We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design Masterclass · Api Design Protocols
Free preview: the first 2 parts of this 8-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design courseread the rest of this lesson in
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design coursein Api Design Protocols
This lesson is part of the full System Design Masterclass. Unlock it along with 769 other lessons for a one-time payment.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime Access18 Foundation lessons are free, no payment needed. Try them now
It is easy and good
“It is really nice and good Please join the course”
Keep-alive keeps one connection open between two parties. Connection reuse takes this further: a pool of pre-established connections shared across your entire application. When a request needs a connection, it borrows one from the pool. When it is done, it returns it. No connection setup, no teardown, no waiting for handshakes.
Database connection pools are the most familiar example. Instead of opening a new Postgres connection for every query (which costs 30-100ms for + TLS + authentication), your app maintains a pool of 20 connections. Each query borrows a connection, executes, and returns it. The pool handles the lifecycle.
connection pools work the same way. An HTTP client maintains a pool of keep-alive connections to each target host. When your code makes a request, the client grabs an idle connection from the pool. If none are available, it either creates a new one (up to the pool limit) or waits.
Node.js's http.Agent, Python's requests.Session, Java's HttpClient, Go's http.Client with Transport.MaxIdleConnsPerHost all maintain connection pools. Understanding pool sizing and configuration is what separates a service that handles 10,000 requests per second from one that falls over at 500.
Application Thread 1 --borrow--> [Connection Pool] --> Backend Service
Application Thread 2 --borrow--> (5 connections)
Application Thread 3 --wait-->
Thread 1 finishes --return--> [Connection Pool]
Thread 3 --borrow (now available)-->
Key pool parameters:
| Parameter | What It Controls | Too Low | Too High |
|---|---|---|---|
| Min connections | Always-ready connections | Cold start for first requests | Wastes idle resources |
| Max connections | Upper limit | Requests queue/fail at high load | Overwhelms backend |
| Idle timeout | When to close unused connections | Frequent reconnection overhead | Resource waste |
| Max wait time | How long to wait for a connection | Premature failures | Slow response under load |
Pool sizing rule of thumb (from the PostgreSQL wiki): Set max_pool_size to roughly (core_count * 2) + effective_spindle_count on the database server. For a 4-core SSD machine, that is about 9 connections. More connections almost never improves because the contention moves from the pool into the database's internal lock manager. HikariCP's documentation puts it bluntly: "A pool of 10 connections serving a database with 10,000 concurrent users will perform better than a pool of 10,000 connections."
Total pool math matters. If Postgres can handle 100 connections across all your app instances, and you have 5 instances, each pool should max at 20, not 100. Five instances times 20 equals 100. If each instance had a pool of 100, you would potentially open 500 connections, five times what the database can handle.