Name: System Design Masterclass
Availability: InStock

Varnish put the cache in front of the application. Hazelcast puts it inside: every application process is a member of the cluster, and the map lives across all of them. There is no cache server to run.

The sentence everybody repeats about this is that the cache is in the same process, so a read is a memory read. That sentence is true for one key in every N.

Two panels. What the word embedded suggests: the cache is in your process, so a read is a memory read and there is no network, with reads like this being 1 in 4 and a measured p50 of 0.017 ms. What four members actually do: the map is partitioned, one member owns each key, and the rest forward to it, with reads like this being 3 in 4 and a measured p50 of 0.033 ms. — ComparisonFigure 1. One read in four was answered by the process that received it, and the rest crossed a socket
Four real member processes, a map partitioned across them by key hash, and 8,000 real reads sent to one of them. 25.9 percent were answered locally, which is one in four because there are four members.

Read the word embedded as being about the deployment, not the access path. There is no separate cache server, and there is still a network hop for every key this particular process does not happen to own.

A hand-drawn figure. Four boxes in a row are labelled member 0 through member 3, under a heading reading four application processes, each one a member of the cluster. A curved red arrow runs from a label reading a read arrives here, under the first box, up to the third box, labelled and the key lives here. A note reads that the read is forwarded over a real socket, and the calling code sees one map lookup either way. — SketchFigure 2. The map is one map, and each key has exactly one owner among the members
Hazelcast hashes the key into one of 271 partitions and assigns partitions to members. A get on a member that does not own the partition is forwarded to the one that does.

A terminal capture of the Hazelcast lab. Section one shows 2,068 reads answered by the member asked at a median of 0.017 ms and 5,932 forwarded to the owning member at 0.033 ms. Section two compares a uniform and a Zipf workload with the near cache off and on: uniform goes from 6,025 forwarded to 4,923, Zipf goes from 5,942 to 1,554 with the median falling from 0.033 ms to 0.017 ms. Section three shows a control with the near cache off measuring 0 stale reads in 39,817, then 0.00, 0.10, 1.36 and 6.65 percent stale as the invalidation flight time goes from immediate to 1, 5 and 25 milliseconds. Section four shows 15,863 of 20,000 keys changing owner when a fifth member joins, which is 79.3 percent against the 20.0 percent that had to. — terminal-figureFigure 3. The measurement. Four member processes, four questions
Every number here is a real socket round trip between real processes. The counts and shares are exact and the latencies are a floor, because loopback removes the network.

A grouped bar chart in milliseconds. Reads answered by the member asked show a median of 0.017 ms and a 99th percentile of 0.043 ms. Reads forwarded to the owning member show a median of 0.033 ms and a 99th percentile of 0.090 ms. — ChartFigure 4. The same map call is two different operations, and the code cannot tell which
A local read is a dictionary lookup. A forwarded read is a socket round trip to another process, and on a real network that is a real network hop, not a loopback one.

These numbers are a floor, and the ratio is what transfers. Loopback removes the network from the forwarded case, so on real hardware the gap between the two bars is wider than this, not narrower.

Hazelcast

The rest of this lesson

Continue this lesson

Embedded Does Not Mean Local

What Four Members Actually Measured