Object Storage

When you PUT an object, the system does not write it to one disk. It splits the object into fragments and spreads copies or erasure-coded pieces across many drives, many servers, and often many data centers in a region. Amazon S3 advertises 99.999999999 percent durability (eleven nines), which it reaches by storing each object redundantly across at least three Availability Zones. Losing a disk, a server, or a whole building does not lose your data.

Erasure coding is the trick that makes this cheap. Instead of keeping three full copies (200 percent overhead), the system breaks an object into, say, 10 data fragments plus 4 parity fragments, scatters all 14 across separate failure domains, and can rebuild the original from any 10 of them. That tolerates multiple simultaneous failures while adding only about 40 percent overhead instead of 200 percent.

Lookups go through a key-to-location index, not a directory walk. Because the namespace is flat and every object is addressed by a single key, the index shards cleanly across thousands of machines, so adding capacity is just adding more nodes. There is no central file allocation table to become a bottleneck, which is how a single bucket can hold billions of objects without slowing down.

The cost of all this is consistency and latency semantics. Reads are eventually or, on modern S3, strongly consistent for the whole object, but you never lock a region of a file the way a database row lock works. First-byte latency is tens to low hundreds of milliseconds, much slower than a local SSD, so object storage is for throughput and capacity, not for transactional hot paths.

Picture a photo-sharing app. A user uploads a 4 MB JPEG. Your backend does a single PUT to a bucket with a key like users/8a3f/original.jpg, attaching metadata such as content-type image/jpeg and a tag tier=hot. The object is immediately redundant across three Availability Zones.

A background job reads that object, generates a 200 KB thumbnail, and PUTs it as users/8a3f/thumb.jpg. When the feed loads, your app does not stream bytes through its own servers. It returns the object URLs, and the browser fetches them directly, usually through a CDN like CloudFront caching the bucket so the actual object store is only hit on a cache miss.

Months later a lifecycle rule notices the original has not been read in 90 days and automatically transitions it to a cheaper infrequent-access or archive tier, cutting its storage cost by half or more with zero code changes. That whole pattern, durable origin plus CDN edge plus automatic tiering, is the standard way large products serve user content at scale.