Memcached

Memcached is a cache that lives entirely in RAM. You give it a key and a value, it holds that value in memory, and the next time you ask for the same key it hands the value back without touching disk or a database. A read from Memcached typically takes well under a millisecond, while the same MySQL or PostgreSQL query it replaces might take 10 to 100 milliseconds.

It was created by Brad Fitzpatrick in 2003 to take load off the database behind LiveJournal. The design goal was deliberate simplicity: store opaque bytes under a string key, with an optional expiration time, and nothing else. There are no data types beyond raw blobs, no persistence to disk, no replication, and no querying. You can only set, get, delete, increment, and decrement.

Values are capped at 1 MB by default and keys at 250 bytes. If a value would be larger than 1 MB you are expected to split it or not cache it. This constraint keeps the memory allocator predictable and the whole system fast.

Each Memcached server holds an independent hash table in memory. The cluster is shared-nothing: servers do not talk to each other and do not know other servers exist. All of the intelligence about which server owns which key lives in the client library, not in the servers.

The client takes the key, hashes it, and uses that hash to pick one server out of the list it was configured with. Most modern clients use consistent hashing so that adding or removing one server only remaps a fraction of the keys instead of shuffling everything. This is why you scale Memcached by simply adding more boxes and more total RAM.

Inside one server, memory is managed with a slab allocator. Memcached carves RAM into pages, splits each page into fixed-size chunks called slabs (for example 96 bytes, 120 bytes, 152 bytes), and stores each item in the smallest slab class that fits. When the cache is full it evicts items using an LRU policy, throwing out whatever was least recently used. Expired and evicted items are simply gone; there is no second tier and no recovery.

Memcached is also heavily multithreaded, so a single instance with many cores can serve hundreds of thousands of requests per second over its simple text or binary protocol on TCP or UDP.

Reach for Memcached when you have a read-heavy workload and a clear set of values that are expensive to compute but cheap to store as flat blobs: database query results, fully rendered HTML fragments, API responses, or login session objects. The classic pattern is cache-aside, where the app checks Memcached first, and on a miss reads the database, then writes the result back into the cache with a short expiry.

Its strength is also its limitation. Because it is just a flat key-value store, there are no lists, sets, sorted sets, counters with atomic guarantees beyond increment, pub/sub, or persistence. If your data vanishes when a node restarts, that has to be fine, because everything is volatile. Memcached is purely a cache, never a source of truth.

The honest comparison is with Redis. Redis has richer data structures, optional persistence, replication, and clustering built in, which is why many new projects start there. Memcached still wins in narrow cases: it is genuinely multithreaded so one large instance uses many cores well, its slab allocator can have lower memory overhead for large numbers of small uniform values, and its smaller feature surface means fewer ways to misuse it. Pick Memcached when you want a dumb, fast, horizontally scalable blob cache and nothing more.

Picture a product page that runs a join across orders and inventory to compute how many units are left. That query takes 40 milliseconds and runs on every page view. Under traffic of 5,000 views per second the database is doing 5,000 of those joins every second and falls over.

With Memcached in front, the app computes the result once, stores it under a key like inventory:product:12345 with a 30 second expiry, and serves the next 150,000 requests from RAM in under a millisecond each. The database now runs that join roughly once every 30 seconds per product instead of thousands of times per second.

The cost you accept is staleness: for up to 30 seconds the displayed count can be wrong. For a stock counter that is usually acceptable, and you can shorten the expiry or delete the key on writes to tighten it. That trade between freshness and load is the core decision in every cache, and Memcached makes it about as simple as it gets.