Name: System Design Masterclass
Availability: InStock

The last lesson ended with a claim: memcached decided differently on nearly every question, and the two designs are both coherent, not one modern and one dated. A claim like that has to be measured or it is a table copied from a comparison post.

So a real memcached and a real redis-server were started side by side on one machine and asked the same three questions.

Two panels. memcached with four threads: a 5 MB read arrives, one thread takes it, three others stay free, with a 99th percentile of 0.021 ms idle and 0.028 ms while busy. Redis with one loop: the same 5 MB read, the only thread takes it, everything else queues, with a 99th percentile of 0.021 ms idle and 0.691 ms while busy. — ComparisonFigure 1. One big reader multiplied the small GET by 1.3 on memcached and by 32 on Redis
Both servers were given the same 5 MB value, the same small key and the same client behaviour. The only difference is that memcached hands each connection to one of four threads.

This is the axis people usually get backwards. memcached is not faster than , and idle they measured the same to three decimal places. It is less affected by what other clients are doing, which is a different property and it only appears under load.

A terminal capture of the memcached lab. A first table of the same small GET against both servers: memcached idle p50 0.015 ms, p99 0.021 ms, worst 0.2 ms; memcached with one big reader 0.016, 0.028, 0.2; redis idle 0.016, 0.021, 0.2; redis with one big reader 0.017, 0.691, 1.3 ms with 3 samples over a millisecond. A second table of memory used per stored value: at 96 bytes memcached uses a 192 byte chunk and Redis 136; at 1,024 bytes memcached uses 1,184 and Redis 1,562; at 60,000 bytes memcached uses 66,232 and Redis 65,563. A third section fills memcached with 20,000 items of 500 bytes then 400 items of 40,000 bytes, and shows class 9 holding 10 pages while class 28 holds 1. — terminal-figureFigure 2. The measurement. Two servers, one machine, three questions each
The big reader multiplied the small GET's 99th percentile by 1.3 on memcached and by 32.4 on , from an idle 0.021 ms on both.

A bar chart of the 99th percentile of the identical small GET in four runs, in milliseconds. memcached idle 0.021, memcached busy 0.028, Redis idle 0.021, Redis busy 0.691. — ChartFigure 3. Three of these four bars are the same height, and the fourth is what one thread costs
Bars one and three are the two servers with nothing else happening. Bar two is memcached under interference, and bar four is Redis under exactly the same interference.

Choose between them on this axis only if your workload actually has commands of wildly different cost. If every operation is a small GET or SET, the two servers measure the same and the decision belongs to the other differences in this lesson.

A hand-drawn figure. Four boxes in a row represent worker threads, with connection lines dropping into each from above, labelled connections spread across the threads at accept time. Three boxes are labelled free and the second is labelled busy for 3 ms. A note reads that a connection is assigned to a thread once and stays there, so which thread you get is decided before you send anything. — SketchFigure 4. Four threads is four queues, and a connection is put in one of them and left there
That is what removed the blocking in the measurement. It is also the reason the protection is partial: the clients sharing a thread with the slow one still wait.

Read the result as a reduced blast radius, not an escape from it. A slow operation on memcached still stalls whoever is behind it on that thread, and the improvement is that the other three quarters of the connections never notice.

Where this shows up in interviews

Where this shows up in interviews

Memcached

The rest of this lesson

Continue this lesson

The Opposite Choice, Measured

What The Two Servers Actually Measured