What A Replacement Tells You
Caffeine is the library that replaced Guava's cache. The interesting question about a replacement is what specifically was wrong with the thing it replaced.
Two of Guava's design decisions are measurable, and the measurements do not agree with the received wisdom about either.
Segmented eviction is the one everybody warns about. Lazy maintenance is the one nobody mentions.
Measure the thing people warn you about before repeating the warning. One of these two is a real operational problem in ordinary configurations, and it is the quiet one.
The Famous Problem, Measured
Guava splits the cache into independent segments, the way a ConcurrentHashMap does, and each segment evicts on its own capacity. A cache configured for two thousand entries is therefore several small caches, and several small caches hold less than one large one.
Every run saw the identical 300,000 request stream, and the counts are exact.
The warning is that several small caches hold less than one large one. That is true, and at ordinary settings the amount less is too small to find.
This is a negative and it is worth publishing as one. Segmented eviction is the first thing anybody says about Guava's cache, and at the concurrency level it ships with, it cost nothing measurable.
The damage is not about the segment count on its own. It is about how few entries each segment ends up holding, and a hundred entries divided sixteen ways is six each.
The folklore is half right. Segmented eviction is invisible at ordinary sizes and becomes a problem on small caches, which is exactly where somebody is most likely to have raised the concurrency level to fix a lock.
Segments exist so a concurrent cache can be written without one lock over everything. A segment that consulted the others before evicting would need exactly the lock segments were introduced to avoid.