Send The Code To The Data
Ignite made queries expensive by moving them across members. Coherence's answer to the same problem is older and blunter: stop moving data, and move the code instead.
An entry processor is a piece of logic the caller ships to whichever member owns the key. It runs there, against the entry, under that member's own lock.
Read, modify and write from the caller is two round trips and measured 0.033 ms. One entry processor is one round trip and measured 0.017 ms.
Halving the round trips halved the time, which is the argument everybody makes for this feature. It is the less important of the two arguments.
The Argument That Actually Matters
Two callers reading the same value, each adding one and writing it back, produce one increment instead of two. Every step succeeded and the result is silently short.
Eight callers, 3,200 increments, and the counter reached 708. The control run through the atomic operation reached exactly 3,200.
The entry processor row is the control: it runs under the owning member's lock, so it is required to reach exactly 3,200, and it did. That is what makes the number beside it real.
2,492 increments of 3,200 are simply gone, which is 77.9 percent of the work. No error was returned, nothing was retried and nothing was logged. An application counting its own successful calls would report 3,200 of them in both runs.
Every read-modify-write against a shared store has this gap. It is invisible in the code, invisible with one caller, and its width is the network you were so pleased to have reduced.
Notice that this failure gets worse as the system gets busier and as the network gets slower, which are the two conditions under which nobody is looking. That is why lost updates are usually found by a customer.