Timeout
A maximum duration to wait for an operation to complete before giving up. Without timeouts, a stalled dependency can hang your entire system indefinitely.
What is Timeout?
In short
A timeout is a hard limit on how long one operation will wait for a response before it gives up and fails. If a network call, database query, or lock acquisition does not finish within the set duration, the caller abandons it and returns an error instead of waiting forever.
What a timeout actually is
A timeout puts a clock on an operation. You start a request, you set a deadline, and if the result has not arrived by then you stop waiting and treat it as a failure. The operation itself might still be running on the other side, but as far as your code is concerned it is done and you move on.
Without a timeout, a single slow or dead dependency can freeze your service. A thread calls a payment API, the API never answers, and that thread sits blocked. Under load you run out of threads or connections, and a problem in one downstream service turns into a full outage in yours. Timeouts convert an unbounded wait into a bounded, predictable failure you can handle.
There are several kinds you usually set separately. A connection timeout caps how long you wait to establish a TCP connection. A read or socket timeout caps how long you wait for bytes after the connection is open. A request or total timeout caps the whole call end to end. Most outages come from setting one and forgetting the others.
How it works under the hood
When you make a blocking call, the runtime arms a timer for the deadline you gave it. If the response comes back first, the timer is canceled and you get your data. If the timer fires first, the runtime interrupts the wait, usually by closing the socket or throwing a timeout exception, and the call returns control to you.
In modern systems the deadline is often passed down the call chain. A request comes in with a 2 second budget, the service spends 300ms, then calls a downstream with the remaining 1.7 seconds, which calls its own downstream with what is left. This is deadline propagation, and gRPC supports it directly. It stops a deep chain from each independently waiting its full timeout and blowing past the original budget.
A timeout on its own does not free the work happening downstream. The server may keep processing a query you already gave up on. That is why timeouts pair with cancellation: pass a cancellation token or context so the downstream can see the caller left and stop wasting CPU, connections, and locks.
Picking the number matters. A common rule is to set the timeout above the high percentile of normal latency, often the p99 plus headroom, not the average. If your p99 is 200ms, a 150ms timeout will fail healthy requests, and a 30 second timeout will let real failures hang far too long.
When to use it and the trade-offs
Set timeouts on every call that crosses a process boundary: HTTP requests, database queries, cache lookups, message broker calls, lock acquisition, even DNS. The default in most HTTP clients is no timeout or a very long one, so the dangerous default is doing nothing.
The main trade-off is too short versus too long. Too short and you fail requests that would have succeeded, especially during normal latency spikes, and your retries pile more load onto a service that is already slow. Too long and a stuck dependency drains your thread pool before the timeout ever fires, which defeats the point.
Timeouts rarely work alone. Pair them with bounded retries plus exponential backoff and jitter so a timed-out call gets a second chance without hammering the target. Pair them with a circuit breaker so that once a dependency is clearly down you stop calling it at all instead of timing out on every request. And always give the user a sensible fallback or error rather than a spinner that never resolves.
A concrete example
Picture a checkout service that calls a fraud-scoring API on every order. Normally the API answers in 80ms. One day its database degrades and it starts taking 45 seconds per call. With no timeout, every checkout thread blocks on that call. Within a minute the checkout service exhausts its thread pool, and now nobody can check out, even though the only sick component was fraud scoring.
Now add a 500ms timeout plus a fallback. After 500ms the call is abandoned, the order is queued for asynchronous fraud review, and the customer completes checkout normally. The fraud outage stays contained to fraud scoring. This pattern, fail fast and degrade gracefully, is exactly how a timeout turns a cascading failure into a minor, invisible blip.
Netflix popularized this approach with Hystrix, wrapping every remote call in a timeout, a fallback, and a circuit breaker so one slow service could never take down the whole streaming experience.
Where it is used in production
gRPC
Every call carries a deadline that propagates down the chain, so a deep call tree shares one budget instead of stacking timeouts.
Netflix Hystrix and Resilience4j
Wrap remote calls in a timeout plus fallback plus circuit breaker so one slow dependency cannot exhaust threads and cascade.
NGINX
proxy_connect_timeout, proxy_read_timeout, and proxy_send_timeout cap how long the proxy waits on upstream servers before returning a 504.
PostgreSQL
statement_timeout and lock_timeout abort queries or lock waits that run too long, freeing connections and preventing pileups.
Frequently asked questions
- What is a good timeout value to use?
- Base it on observed latency, not a guess. Set it above your p99 latency with some headroom, so healthy requests almost never trip it but real failures fail fast. If p99 is 200ms, something like 500ms is reasonable. Avoid copying a default like 30 seconds, which is far too long for most internal calls.
- What is the difference between a connection timeout and a read timeout?
- A connection timeout limits how long you wait to establish the TCP connection to the server. A read timeout limits how long you wait for data after the connection is open. You should set both, because a server can accept your connection quickly and then stall while processing, and only the read timeout catches that.
- Do timeouts and retries work together?
- Yes, and they are usually used together. The timeout bounds each attempt, and a bounded retry with exponential backoff and jitter gives a timed-out call another chance. Be careful: retrying every timeout immediately can amplify load on an already struggling service, so cap retries and add backoff.
- Does a timeout stop the work on the server?
- Not by itself. The server may keep running the query or request you gave up on, holding connections and locks. To actually stop downstream work you need cancellation, such as a cancellation token or a gRPC or context deadline that the downstream checks and honors.
- What is the difference between a timeout and a circuit breaker?
- A timeout bounds a single call so it cannot hang forever. A circuit breaker watches the recent failure rate across many calls and, once a dependency looks unhealthy, stops sending requests to it entirely for a cooldown period. Timeouts handle one slow call; circuit breakers stop you from making thousands of doomed calls.
Learn Timeout hands-on
This page explains the idea. The full lesson lets you step through the ring as servers join and leave, read the implementation, and check yourself with a quiz. It is one of 760+ lessons in the System Design Masterclass, from your first API call to distributed consensus. Eleven Foundation lessons are free, no signup. Lifetime access is ₹499 in India or $7.99 worldwide, one payment, no subscription.
Related lessons
Lessons that touch on Timeout as part of a larger topic.
Connection Timeout
Maximum time a client waits to establish a connection before giving up
foundation · core fundamentals
Request Timeout
Maximum time a client waits for a server to send a complete response
foundation · core fundamentals
Timeout Patterns
Set appropriate timeouts for every external call, deadline propagation and timeout budgets
advanced · reliability resilience
Session Windows
Dynamic windows based on activity gaps, group events by user sessions with configurable timeouts
advanced · stream batch processing
See also
Related glossary terms you might want to look up next.
Circuit Breaker
A pattern that stops calling a failing service after repeated failures, preventing cascade failures. Like an electrical circuit breaker that cuts power to prevent fires.
Retry
Automatically re-attempting a failed operation, usually with exponential backoff. Essential for handling transient failures in distributed systems.
Latency
The time delay between sending a request and getting a response. Amazon found every 100ms of extra latency costs 1% in sales.
Microservices
An architecture where an application is split into small, independent services that communicate over the network. Each service owns its own data and can be deployed separately.
Monolith
A single, unified application where all features share the same codebase and deployment. Simpler to start with but harder to scale individual parts.
Service Discovery
The mechanism by which microservices find and communicate with each other. Services register themselves and others can look them up by name.