Last lesson, you learned about connection timeouts, the time to establish a connection. The handshake. The "hello, are you there?"
But the key point is that establishing a connection and getting an answer are two very different things. You can call a restaurant, someone can pick up the phone, and then put you on hold for 45 minutes. The connection was established instantly. The response never came.
A request timeout is the maximum time your client waits for the server to send back a complete response after the connection is already established.
Here's a real scenario I've seen multiple times: a backend service accepts a connection just fine, the handshake completes in 2ms. But then the service tries to run a database query that takes a lock, and the lock is held by another query, and that query is waiting on yet another lock. Deadlock. The service never responds. Without a request timeout, your client sits there forever, connected to a server that has absolutely nothing to say.
This is more insidious than a connection failure. At least when a connection fails, you get an error quickly. A service that connects but never responds is a silent killer.
Once the connection is established, the actual request-response cycle begins:
The request timeout (also called read timeout or socket timeout depending on the library) covers steps 2 through 5. It starts ticking the moment your client finishes sending the request and stops when the complete response is received.
| Timeout | Typical Range | Why |
|---|---|---|
| Connection timeout | 1-5 seconds | TCP handshake is fast. If it doesn't happen quickly, the server is unreachable. |
| Request timeout | 5-30 seconds | Server legitimately needs time to process. A database query, an ML inference, a third-party API call, these take real time. |
A connection timeout of 2 seconds with a request timeout of 15 seconds is a very common configuration. It says: "I expect to connect within 2 seconds, and I'll give the server up to 15 seconds to respond."
Watch how one slow database query, without a request timeout, can take down an entire system. Then see how adding timeouts prevents the cascade.
Setting request timeouts requires understanding what the downstream service actually does.
| Operation Type | Reasonable Request Timeout | Example |
|---|---|---|
| Simple read (cache hit, key lookup) | 1-3 seconds | GET /users/123 |
| Standard CRUD with database | 5-10 seconds | POST /orders |
| Complex query or aggregation | 10-30 seconds | GET /analytics/report |
| File upload/download | 30-120 seconds | POST /files/upload |
| Long-running job (synchronous) | Don't. Use async instead. | Report generation |
When a request times out, the natural instinct is to retry. Usually that's fine. But there's a trap.
Imagine Service A times out calling Service B after 5 seconds. Service A retries. Meanwhile, Service B is actually just overloaded, it's processing requests slowly but not dead. The retry adds to Service B's queue. A hundred other clients are doing the same thing. Now Service B is getting twice the traffic, the original requests plus all the retries. It gets even more overloaded. Timeouts increase. More retries fire. More load. The system spirals into collapse.
This is called a retry storm, and it's one of the most common causes of prolonged outages.
3 questions - Score 80% to pass
What is the key difference between a connection timeout and a request timeout?
Service A (timeout: 10s) calls Service B (timeout: 12s to Service C). What's wrong with this setup?
A service is overloaded and responding slowly. Your client times out and retries immediately. 500 other clients do the same. What happens next?
I've seen codebases that set a global 30-second timeout for all HTTP calls. This means a simple health check endpoint, which should respond in 50ms, gets a 30-second grace period before failing. If that health check hangs, your won't know the service is unhealthy for 30 seconds. That's 30 seconds of sending traffic to a dead instance.
Different operations deserve different timeouts. Your health check might get 2 seconds. Your payment processing might get 15 seconds. Your report generation endpoint should be async, not synchronous with a long timeout.
In architectures, request timeouts need to account for the full call chain.
Say Service A calls Service B, which calls Service C:
Service B's timeout must be less than Service A's. If Service B waits 12 seconds for Service C, but Service A already gave up after 10, Service B is doing useless work, the caller already moved on. The response from Service C arrives, gets passed back to B, which passes it to... Nobody. Service A hung up 2 seconds ago.
Rule of thumb: Each hop in the chain should have a timeout that's 70-80% of the caller's timeout, leaving room for network transit and processing overhead.
Exponential backoff, don't retry immediately. Wait 1 second, then 2, then 4, then 8. Give the struggling service time to recover.
Jitter, add randomness to your backoff. If 1,000 clients all retry at exactly 1 second, you get another spike. Add random delay (e.g., 1 second + random 0-500ms) to spread retries out.
Max retries, cap it at 2-3 retries. If a request fails 3 times, it's not going to succeed on the 4th. Fail and tell the user.
Only retry operations, retrying a GET is always safe. Retrying a POST that creates a payment? You might charge someone twice. (We'll cover idempotency in a later lesson.)