TCP was designed in 1981 for networks with kilobit bandwidth and high packet loss. The default settings reflect that era: conservative congestion windows, slow ramp-up, aggressive timeouts. On modern high-bandwidth, low-loss cloud networks, these defaults waste enormous capacity.

A fresh TCP connection starts with an initial congestion window (initcwnd) of 10 segments, roughly 14 KB. That means the first burst of data your server can send is only 14 KB, regardless of whether the link can handle 10 Gbps. It takes multiple round trips to ramp up to full throughput. For a typical HTTP request that transfers 50 KB total, the connection finishes before TCP ever reaches cruising speed.

Google quantified this in their 2011 paper "An Argument for Increasing TCP's Initial Congestion Window." They found that increasing initcwnd from 3 to 10 reduced HTTP latency by 10% on average. For web pages, where the critical rendering path fits in the first few round trips, this is the difference between a 200ms and a 220ms page load.

TCP optimization is not about changing the protocol itself. It is about tuning parameters to match your actual network conditions. Cloud networks between instances in the same region have 0.5ms RTT and near-zero packet loss. The TCP defaults were designed for 200ms RTT links with 2% loss. Closing that gap can improve throughput by 2-10x.

The congestion control algorithm decides how fast TCP sends data. This single parameter has more impact on throughput than almost any other setting.

Cubic (Linux default since 2006): Loss-based. Increases the sending rate until a packet is dropped, then backs off dramatically. Problem: on a 10 Gbps link with even 0.1% random packet loss (common on long-distance links), Cubic stays permanently underutilized because it interprets every loss as congestion.

BBR (Google, 2016): Model-based. Directly estimates the bottleneck bandwidth and minimum RTT by probing the network. Sends data at the estimated optimal rate regardless of occasional random loss. Google deployed BBR on youtube.com and saw throughput increase by 4% globally and by 14% in developing countries with lossier networks.

DCTCP (Microsoft, 2010): ECN-based. Uses Explicit Congestion Notification signals from switches instead of waiting for packet loss. Reacts proportionally to congestion level rather than the binary loss/no-loss signal. Designed specifically for data center networks with ECN-capable switches. Reduces tail latency by 3-5x compared to Cubic in data center environments.

BBRv2 (Google, 2019+): Addresses BBRv1's fairness problems. BBRv1 was too aggressive when sharing bandwidth with Cubic flows. BBRv2 incorporates loss signals alongside bandwidth estimation. Still experimental but deployed on Google's internal network.

1. Increase Initial Congestion Window:

# Linux: set initcwnd to 20 (default is 10)
ip route change default via 10.0.0.1 initcwnd 20

Allows 28 KB in the first burst instead of 14 KB. Most CDNs (Cloudflare, Fastly, Akamai) run initcwnd=30 or higher.

2. Enable TCP Fast Open (TFO):

# Linux: enable TFO for both client and server
echo 3 > /proc/sys/net/ipv4/tcp_fastopen

Normal TCP requires a full three-way handshake before data flows. TFO lets the client include data in the SYN packet on repeat connections using a cryptographic cookie. Saves one full round trip. Google found TFO reduced HTTP transaction latency by 15% on Chrome.

3. Window Scaling:

# Linux: enable window scaling (usually on by default)
echo 1 > /proc/sys/net/ipv4/tcp_window_scaling

Without window scaling, the maximum receive window is 64 KB. On a 10 Gbps link with 50ms RTT, the bandwidth-delay product is 62.5 MB. A 64 KB window can only use 0.1% of the link. Window scaling extends the maximum to 1 GB.

4. TCP Keep-Alive tuning:

# Send keepalive probe after 60s idle (default: 7200s = 2 hours)
echo 60 > /proc/sys/net/ipv4/tcp_keepalive_time
# Probe interval: 10s
echo 10 > /proc/sys/net/ipv4/tcp_keepalive_intvl
# Give up after 6 failed probes
echo 6 > /proc/sys/net/ipv4/tcp_keepalive_probes

The default 2-hour keepalive timer means dead connections sit around for 2 hours consuming server resources. For microservices behind load balancers, 60 seconds is far more practical.

5. Nagle's Algorithm (disable for latency-sensitive):

# Disable Nagle's algorithm on the socket
setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, &one, sizeof(one));