Data Lifecycle Management

Your application generates 10GB of data per day. After a year, that's 3.6TB. After five years, 18TB. Storing 18TB on high-performance SSDs costs $3,600/month on AWS (EBS gp3 at $0.08/GB). But the key point is that 90% of that data hasn't been accessed in over a year.

Last month's analytics events are queried daily. Last year's events are queried monthly. Three-year-old events are queried once a year during audits. Storing all of it on the same tier is like renting a downtown penthouse to store boxes you open once a year.

Data lifecycle management moves data through storage tiers as it ages: hot data on fast storage, warm data on cheaper storage, cold data on archival storage, and expired data deleted entirely. The principle is simple: match storage cost to access frequency. Spotify's data platform team reported saving $2.4M per year by implementing automated lifecycle policies on their 500PB data lake. Netflix moves petabytes of encoding artifacts to Glacier within 48 hours of video launch, saving millions annually.

The economics at scale: Moving 10TB from hot ($800/month) to cold ($40/month) saves $760/month, which is $9,120/year. For companies with petabytes, lifecycle management saves millions. Dropbox reported saving over $75M by moving cold data to their own cold storage tier (Magic Pocket) instead of keeping it on AWS S3.

Standard lifecycle rules:

• After 30 days: Move from hot to warm (75% cost reduction).
• After 90 days: Move from warm to cold (97% cost reduction from hot).
• After 7 years: Delete or move to frozen if legal hold exists.
• These rules are defined once in S3 lifecycle policies, Azure lifecycle management, or GCP object lifecycle rules. No cron jobs. No custom code.