We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
This idea carries a full system design question on its own. Each walks through the full answer.
System Design Masterclass · Stream Batch Processing
Free preview: the first 2 parts of this 6-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design courseread the rest of this lesson in
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design coursein Stream Batch Processing
This lesson is part of the full System Design Masterclass. Unlock it along with 769 other lessons for a one-time payment.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime Access18 Foundation lessons are free, no payment needed. Try them now
It is easy and good
“It is really nice and good Please join the course”
Tumbling windows give you one number per bucket. A 5 minute tumbling window produces a new count every 5 minutes. For a metrics dashboard, that is fine. For Lyft's surge pricing, it is an eternity. Demand shifts by the minute on a Friday night when a stadium empties. A 5 minute refresh cadence means a driver 2 miles away never gets the surge signal before it is over.
Hopping windows solve this by sliding a fixed size window forward at a smaller interval. A 5 minute window with a 1 minute hop produces a new 5 minute average every single minute. You get the smoothing of the longer window combined with the freshness of the shorter hop. Lyft uses exactly this shape on geographic hex grid cells to feed their pricing model. Uber's demand heatmap uses a similar overlapping window with a 30 second hop to catch concert let-outs and sporting events within a minute.
The price is literal. Every event that falls within the window is written to S/H overlapping buckets in the state backend, where S is the window size and H is the hop. A ratio of 5 means five state writes per arriving record. That write amplification is the entire operational story of hopping windows and the single thing you must size correctly before deploying one.
By the end of this lesson you will understand why hopping windows exist, how the overlap ratio drives memory cost, and how to pick the right size and hop for your workload without melting the state backend.
Before going deeper into hopping mechanics, it helps to see all three fixed width window shapes side by side. The differences are small in definition but large in memory cost and output cadence.