Loading...
System Design and AI Engineering, by Roni Das. 980 interactive lessons.
Start LearningSystem design interview guide
Short answer
A time-series database stores each series (one metric name plus labels such as region=eu) as points in time order. New points go to a write-ahead log, a file on disk that only grows at the end, and to memory. Every two hours the oldest data is written out as a compressed block that never changes. Timestamps are stored as the change in the gap between them, and values as only the bits that changed: about 1.37 bytes per point in Facebook's Gorilla paper. A label index finds series, summaries called rollups serve long time ranges, and old data is deleted a whole block at a time.
in this guide
8 parts open to everyone · 3 with a free account · 3 with the course
You cannot pass this question by naming a product. The interviewer wants to see you start from the shape of the data and reach the design yourself. Three facts describe the data. It is written once and never changed. Writes arrive all the time, in huge numbers. Reads are mostly about recent time. From these facts, a good candidate explains three things. First, why a normal row-by-row table struggles. Second, why each series should be stored together and compressed. Third, why splitting storage by time makes deleting old data cheap. The question also has one trap that shows real experience: cardinality. A candidate who raises it without being asked, and explains the defences, makes a strong impression. Finally, there are real trade-offs with no single right answer, such as push versus pull collection, or keeping raw data versus keeping only summaries.
This page is the free part.
₹499 in India$49 everywhere elseonce, for the whole course
The System Design course covers the Time-Series Database design across a run of lessons, not one page. These 4 alone are about 118 minutes of step-by-step reading, every one with a quiz.
Live from the course bench
Real interview questions, live diagrams and measured lab results, pulled at random from the lessons. A new set every time you come back.
Always clarify these in the first 5 minutes of the interview. Do not start drawing boxes until both lists are agreed.
This guide is built on these 6 lessons. Each one teaches one piece of the Time-Series Database design above, step by step.
Lifetime access for ₹499 in India or $49 elsewhere. Interactive diagrams, quizzes, and 20 capstone projects to practise on.
This one question is not enough for the interview.
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (268 diagrams in all). A PDF you keep, junior to staff level.
Two courses by the author of this page:
770 lessons · 18 free to read
₹499 in India · $49 elsewhere, once
Get System DesignYou own this course
210 lessons · 10 free to read
₹999 in India · $49 elsewhere, once
Get AI EngineeringYou own this course
Save your progress and get the interview checklist PDF
A time-series database stores numbers that arrive with a time attached: CPU use every 15 seconds, the temperature of a machine every second, the price of a share many times a second. Take 20,000 servers, each sending 500 such numbers every 15 seconds. That is already about 670,000 new points every second, close to 58 billion points a day. Nobody changes those points later. People mostly ask about the last few hours. Almost every design choice on this page comes from these three facts.
In a time-series database, the basic unit is a series. A series is one metric name plus a set of labels. A label is a name and a value that says where the number came from, such as region=eu. So http_requests_total with service=checkout and region=eu is one series. Each series is a list of (time, value) pairs in time order. New points are first written to a write-ahead log: a file on disk that only grows at the end, so a crash cannot lose them. They are also kept in memory, already compressed. Every two hours, the oldest data in memory is written out as a file that never changes again. Compression is what makes the design affordable. Timestamps are stored as tiny differences, and each value is stored as the bits that changed since the last value. Facebook's Gorilla paper took points from 16 bytes to an average of 1.37 bytes this way. An inverted index, a lookup table from each label to the series that carry it, finds the series a query needs. Old data is deleted one whole file at a time. Long-range charts read pre-computed summaries called rollups. What breaks these systems is cardinality: the number of different series. One label with too many values, like a user id, can turn thousands of series into millions.
Where it shows up
This question comes up for backend and infrastructure roles. It also hides inside bigger questions: design a monitoring system, design an alerting system, design a platform for IoT data (data from internet-connected devices such as sensors), design a stock-price chart service. Several large companies built their own time-series store and wrote about it: Facebook built Gorilla, Google built Monarch, and Uber built M3. Prometheus, InfluxDB, TimescaleDB and ClickHouse are open-source systems you can name and compare.
Spec sheeta Time-Series Database
worked through below, with the maths
The hard parts of this design are taught step by step in the System Design course: Time-Series Databases, Design a Metrics/Monitoring System and LSM Trees.
Databases built for timestamped data, metrics, IoT sensors, financial ticks, and observability.
Design a metrics collection and monitoring system - time-series databases, aggregation pipelines, alerting, and dashboards at scale.
How LSM trees turn random writes into fast sequential writes. Covers the write path, levels, size-tiered vs leveled compaction, and the read cost.
Store data by column instead of by row, the secret behind modern analytics engines.
all part of
System Design Masterclass
770 lessons · about 283 hours · 18 free to read
₹499in India, by UPI
$49everywhere else, by PayPal
You own this course
Continue with Time-Series Databasespay once,
yours for life
Show your math. Pulling numbers from thin air signals you have not thought about the load.
Active series
10 million
Assume 20,000 servers, each exposing 500 series (CPU per core, memory, disk, request counts per endpoint and status). 20,000 x 500 = 10,000,000 active series. These are assumptions for the interview. Say them to the interviewer before you use them.
Points per second
about 670,000
Each series gets one new point every 15 seconds. 10,000,000 / 15 = 666,667 points per second. Per day: 666,667 x 86,400 seconds = about 57.6 billion points.
Disk per day
about 79 GB, not 922 GB
A raw point is a 64-bit timestamp plus a 64-bit number: 16 bytes. 57.6 billion x 16 bytes = 922 GB a day. With Gorilla-style compression at 1.37 bytes per point (the paper's measured average): 57.6 billion x 1.37 = 79 GB a day. Prometheus says it stores 1 to 2 bytes per sample (a sample is one point) on average, so 58 to 115 GB a day is a fair range. The index adds a little on top.
15 days of raw data
about 1.2 to 1.7 TB per copy
Prometheus gives the formula: disk = retention seconds x points per second x bytes per point. 15 days is 1,296,000 seconds. 1,296,000 x 666,667 x 1.37 = 1.18 TB. At 2 bytes per point it is 1.73 TB. Keep 3 copies on 3 machines (replicas) and the cluster needs 3.5 to 5.2 TB.
Memory for active series
about 40 GB per copy
Every active series keeps its newest chunk (a small compressed run of its points), its labels and its index entries in memory. Assume about 4 KB per series. This is an assumption; measure your own system. 10,000,000 x 4 KB = 40 GB, or 120 GB with 3 copies. Spread over 8 machines, each holds about 15 GB.
One-year chart, raw vs rollup
2.1 million vs 8,760 points per series
A year at one point every 15 seconds is 365 x 86,400 / 15 = 2,102,400 points per series. One-hour rollups give 365 x 24 = 8,760 buckets, 240 times fewer. A bucket can hold 5 numbers (count, sum, min, max and a counter value), but a query reads only the ones it needs. Even if it read all 5, that is 43,800 numbers instead of 2.1 million: 48 times less.
Draw six boxes. The first box is the sources: servers, apps or devices that produce points. The second is a distributor, the front door. It receives points, checks them and applies limits, such as a cap on new series for each tenant (one customer or team sharing the database). Points can arrive by push, where sources send data in, or by pull. With pull, the database scrapes each source: it visits a web address on the source on a timer, every 15 seconds in our example, and reads its current numbers. Prometheus works by pull. The distributor then hashes the series name and labels. A hash turns them into a number, and that number picks the machine that owns the series. The third box is the storage machines, often called ingesters. Each one owns some series completely and keeps copies of other machines' series as replicas. The fourth box is object storage, such as Amazon S3. It is cheap, very large storage for files, and it holds the old blocks. The fifth box is the query tier. It sends a query to every machine that owns matching series and adds their answers together. The sixth box is the background jobs: compaction (merging small blocks into bigger ones), rollups and retention. Inside one storage machine, a new point goes two places. It is appended to the write-ahead log (WAL) on disk, so it can be rebuilt after a crash. And it is added to the head, the in-memory part that holds the newest data of every active series. The head compresses points as they arrive. When the head holds about three hours of data, Prometheus writes the oldest two hours out as a block. A block is a folder with the compressed samples, an index and some metadata. It is never edited after that. The WAL files it covered are then deleted, and blocks move on to object storage. The idea behind the whole layout is simple. Writing a point and reading recent data must be fast and local. Every expensive job runs in the background.
In a real interview, sketch this on the whiteboard before diving into any single box.
Follow a single point through the system. This is a good order to explain it in an interview.
The database scrapes the server's /metrics page every 15 seconds (pull), or an agent on the server sends the point in (push). The point is http_requests_total{service="checkout", region="eu"} = 18,204 at 10:00:15.
The distributor sorts the labels, so the same labels always mean the same series, and hashes them. The hash picks the machines that own this series: one main copy and its replicas.
Each owning machine looks the series up. If it is new, the machine gives it a local id and new index entries, and it counts against the tenant's series limit.
The machine appends the point to its write-ahead log. The write is confirmed to the sender once enough replicas have it, for example 2 of 3.
The point is compressed into the series' open chunk in the head. A dashboard asking about the last few minutes reads it from memory.
When the head holds about three hours, the oldest two hours become one block on disk. The WAL files it covered are deleted.
Compaction merges small blocks into larger ones. A rollup job writes 5-minute and 1-hour summaries for long-range charts.
A dashboard asks for the error rate per region. The index finds the matching series, each machine computes its part, and the query tier adds the parts together.
After 15 days the raw block is deleted as one folder. The 1-hour summary of it is kept for a year or more.
Walk through each service. The interviewer wants to hear what each one owns, not just the names.
Receives points and checks them. It sorts each point's labels so the same labels always mean the same series. It applies series limits and relabeling rules, which drop or rewrite labels before storage. Then it hashes the series and sends the point to the machines that own it. It stores nothing itself, so you add more distributors when traffic grows.
A file on disk where every point is appended as it arrives. Adding to the end of a file is much cheaper than writing to random places on disk. After a crash, the machine replays the WAL to rebuild its memory. Prometheus writes its WAL in 128 MB segments and deletes old segments once their data is safely in a block.
The part of each machine that holds the newest data for every active series. It is called the head because it is the newest end of the data. Points are kept as open chunks: small compressed runs of points for one series. Prometheus aims for about 120 samples per chunk. Memory use grows with the number of active series, so cardinality decides how much memory you need.
Compressed files on disk, each covering a fixed time range. Immutable means they are never edited after they are written. In Prometheus a block is a folder with the chunks, an index file and a metadata file. Because blocks never change, they are safe to cache, copy to object storage, merge and delete as a whole. A delete request does not edit a block; it writes a small separate file that marks the deleted data.
A lookup table from each label pair, such as region=eu, to a sorted list of the ids of every series that has it. That list is called a posting list. A query for service=checkout AND region=eu takes the two lists and keeps the ids that are in both. No sample data is read until the series are known.
Background jobs. Compaction merges small blocks into bigger ones. Prometheus merges up to 10% of the retention time or 31 days, whichever is smaller. Rollups (also called downsampling) write a summary for each time bucket, for example every 5 minutes and every hour. Then a one-year chart does not need to read two million points per series.
Deletes data that is past its retention period. Because data is split by time, it deletes whole blocks, which costs almost nothing. Raw data and each rollup level can have different periods.
Finds series with the index, reads the right time range from the head and the blocks, and runs time-series functions: rate over a counter, sum by a label, percentiles. Each machine does as much of the work as it can, so only small partial results travel over the network. It also protects itself with limits, such as a maximum number of series or samples one query may touch.
series. One row per unique metric name plus label set. This is the narrow design used by Prometheus: one series holds one metric, and its labels are stored once for its whole…
Every table for a Time-Series Database, its keys, and which store fits it and why.
The schema: narrow vs wide tables, labels and cardinality. There are two common layouts for time-series data, and an interviewer often asks you to choose. In a narrow layout, each series holds one metric. You keep a series table…
8 deep dives on the parts that break at scale, with 8 diagrams: The schema: narrow vs wide tables, labels and cardinality; Why a normal row-store table struggles; The write path: WAL, memory, and immutable blocks.
Narrow (one metric per series) vs wide (many metrics per row). Narrow is flexible: any source can send any metric, and a new metric needs no table change. Wide stores one timestamp per row and reads related metrics together, which…
7 decisions an interviewer expects you to defend, each with the alternative and why it loses here.
What happens when someone adds a user_id label?
7 follow-up questions with model answers, the interview scorecard for this question, and a mock interview on Design a Time-Series Database with the AI tutor.
Every design on this page comes from a system that was built and written about. Facebook's Gorilla (VLDB 2015) kept the most recent 26 hours of data in memory and took points from 16 bytes to an average of 1.37 bytes. Compared with its earlier store on HBase (a large key-value database), it answered queries 73 times faster. Facebook was adding about 700 million points a minute at the time. Prometheus groups samples into two-hour blocks and protects the head with a WAL in 128 MB segments. It keeps 15 days by default and stores 1 to 2 bytes per sample on average. Its documentation says that local storage is not clustered or replicated. Thanos adds long-term storage around Prometheus, with 5-minute and 1-hour downsampled blocks and a separate retention for each. InfluxDB's TSM engine writes to a WAL and an in-memory cache, then compacts the cache into TSM files (Time-Structured Merge). Its TSI index lives on disk to handle high cardinality. InfluxDB 3 is built on Apache Arrow, DataFusion and Parquet, open-source tools for columnar data. TimescaleDB turns a PostgreSQL table into a hypertable split into time chunks, 7 days each by default. It compresses old chunks with delta-of-delta, XOR and other methods, and keeps rollups up to date with continuous aggregates (rollups it refreshes automatically). ClickHouse provides DoubleDelta and Gorilla codecs, partitions and TTLs on its MergeTree tables. Google's Monarch (VLDB 2020) keeps close to a petabyte (a million gigabytes) of compressed time series in memory, takes in terabytes per second and answers millions of queries per second. It runs in regional zones under a global layer. Uber's M3 uses a Gorilla variant at 1.45 bytes per point and hashes series into 4,096 virtual shards. Across all of them, two lessons repeat. The number of series is the real limit, more than the number of points. And compression plus splitting by time is what keeps the cost down.
Sources
If any of these topics are fuzzy, the interviewer will catch it. Each lesson is 15 to 60 minutes with diagrams, code, and a quiz.
Time-Series Databases
intermediate / database types storage
Design a Metrics/Monitoring System
capstone / capstone
LSM Trees
intermediate / database types storage
Columnar Storage
intermediate / database types storage
Inverted Index
intermediate / database types storage
Write-Ahead Logging
advanced / consistency models
Data Compression
advanced / consistency models
Range Partitioning
foundation / database fundamentals
Consistent Hashing
intermediate / data replication distribution
Tiered Storage
intermediate / database types storage
Prometheus
intermediate / observability monitoring
Histogram Metrics
intermediate / observability monitoring
Time-Series Metrics
intermediate / observability monitoring
interview readiness
Three questions from this guide. Answer them without scrolling up, the way you would have to in the room.
India: UPI, cards or netbanking through Razorpay. Elsewhere: card or PayPal. Download at once from My packs.
Free PDF · 79 pages
25 real tech interview questions, answered in full
Five from each of our AI, GenAI, Inference, Maths for ML and Forward Deployed packs. Each with a diagram and a model answer. Free by email.
It is really nice and good Please join the course