Object Storage

Most people first learn storage as either files or disks. A file system gives you folders, file names, and the ability to open a file and overwrite a few bytes in the middle. Block storage hands you raw fixed-size chunks (blocks) that a database or operating system stitches into something useful. Object storage is a third model. You hand it a blob of bytes and it gives back a unique key. Later you fetch the whole thing by that key.

Each object has three parts: the data itself (an image, a video, a backup, a log file), metadata describing it (content type, size, custom tags like owner or retention class), and a globally unique identifier. There are no nested directories under the hood. Objects live in a flat container called a bucket, and the slashes you see in a key like reports/2026/q2.pdf are just part of the name, not real folders.

You talk to it over an HTTP API, not a file path. The basic operations are PUT to write, GET to read, DELETE to remove, and HEAD to check metadata. There is no append and no partial overwrite. If you change one byte, you upload the whole object again as a new version. That single constraint is what unlocks the scale and durability everything else depends on.

When you PUT an object, the system does not write it to one disk. It splits the object into fragments and spreads copies or erasure-coded pieces across many drives, many servers, and often many data centers in a region. Amazon S3 advertises 99.999999999 percent durability (eleven nines), which it reaches by storing each object redundantly across at least three Availability Zones. Losing a disk, a server, or a whole building does not lose your data.

Erasure coding is the trick that makes this cheap. Instead of keeping three full copies (200 percent overhead), the system breaks an object into, say, 10 data fragments plus 4 parity fragments, scatters all 14 across separate failure domains, and can rebuild the original from any 10 of them. That tolerates multiple simultaneous failures while adding only about 40 percent overhead instead of 200 percent.

Lookups go through a key-to-location index, not a directory walk. Because the namespace is flat and every object is addressed by a single key, the index shards cleanly across thousands of machines, so adding capacity is just adding more nodes. There is no central file allocation table to become a bottleneck, which is how a single bucket can hold billions of objects without slowing down.

The cost of all this is consistency and latency semantics. Reads are eventually or, on modern S3, strongly consistent for the whole object, but you never lock a region of a file the way a database row lock works. First-byte latency is tens to low hundreds of milliseconds, much slower than a local SSD, so object storage is for throughput and capacity, not for transactional hot paths.

Reach for object storage when data is large, read more than written, and rarely edited in place: media files, user uploads, static website assets, data lake tables in Parquet, machine learning training sets, database backups, and log archives. It is the cheapest durable home for cold data, often a few cents per gigabyte per month, dropping below one cent for archive tiers like S3 Glacier.

Do not use it as a database or as a live file share. There is no rename, no partial write, no record locking, and listing a bucket with millions of keys is slow and paginated. If your app needs low-latency random reads and writes inside a file, a POSIX file system or block volume is the right tool. If you need transactions and queries, use a real database and keep only the large blobs in object storage.

The other practical trade-offs are request cost and small-object overhead. You pay per request as well as per gigabyte, so an app that does millions of tiny GETs can run up a surprising bill. Many objects also carry per-object metadata overhead, so storing billions of 1 KB files is inefficient. The common fix is to batch small records into larger objects and put a CDN in front of frequently read ones.

Picture a photo-sharing app. A user uploads a 4 MB JPEG. Your backend does a single PUT to a bucket with a key like users/8a3f/original.jpg, attaching metadata such as content-type image/jpeg and a tag tier=hot. The object is immediately redundant across three Availability Zones.

A background job reads that object, generates a 200 KB thumbnail, and PUTs it as users/8a3f/thumb.jpg. When the feed loads, your app does not stream bytes through its own servers. It returns the object URLs, and the browser fetches them directly, usually through a CDN like CloudFront caching the bucket so the actual object store is only hit on a cache miss.

Months later a lifecycle rule notices the original has not been read in 90 days and automatically transitions it to a cheaper infrequent-access or archive tier, cutting its storage cost by half or more with zero code changes. That whole pattern, durable origin plus CDN edge plus automatic tiering, is the standard way large products serve user content at scale.