Request Batching

You need milk, bread, eggs, butter, and cheese. Do you make five separate trips to the grocery store, one for each item? Of course not. You make one trip and buy everything at once.

Request batching applies the same logic to API calls. Instead of sending 100 individual requests to create 100 users, you send one request containing all 100 users. One HTTP connection, one authentication check, one round trip, one response.

Every request has overhead: DNS lookup, TCP handshake, TLS negotiation, HTTP headers, authentication, parsing. When you send 100 individual requests, you pay that overhead 100 times. When you batch them into one request, you pay it once. At scale, the difference is enormous, both in latency and server load.

Facebook's Graph API supports batching up to 50 operations in a single request. Google's API supports batch requests across multiple endpoints. Stripe supports bulk creation of payment intents. It's a universal optimization.

Individual requests (inefficient):

POST /api/users { "name": "Alice" } → 201 Created
POST /api/users { "name": "Bob" } → 201 Created
POST /api/users { "name": "Charlie" } → 201 Created
// 3 round trips, 3x overhead

Batched request (efficient):

POST /api/users/batch
{
 "operations": [
 { "name": "Alice" },
 { "name": "Bob" },
 { "name": "Charlie" }
 ]
}
→ 200 OK
{
 "results": [
 { "status": 201, "data": { "id": "1", "name": "Alice" } },
 { "status": 201, "data": { "id": "2", "name": "Bob" } },
 { "status": 422, "error": "Duplicate email" }
 ]
}

Key design decisions:

• Atomicity: Should the batch be all-or-nothing (transaction), or should each item succeed/fail independently? Most APIs use independent, partial success is returned with per-item status codes.
• Size limits: Set a maximum batch size (e.g., 100 items). Unbounded batches can time out or consume excessive memory.
• Error handling: Each item in the batch should have its own status. A failure in item 3 shouldn't prevent items 1, 2, and 4 from succeeding.