CPU Optimization

A Go checkout service at an e-commerce company ran 18 pods in production, each requesting 4 CPU cores. At peak traffic, every pod sat at 85 percent CPU and the autoscaler was adding more. The monthly compute bill for this single service was $14,200.

After a 60 second perf record session and a flame graph, the team found two functions consuming 60 percent of all CPU cycles: json.Unmarshal at 42 percent and TLS handshake at 18 percent. The JSON function was being called three times per request because three middleware layers each parsed the body independently. The TLS overhead came from not reusing connections to a downstream payment API.

Two fixes, each under 20 lines of code, dropped peak CPU from 85 percent to 52 percent. The team scaled down from 18 pods to 11 and saved $5,300 per month. The whole investigation took an afternoon.

CPU optimization is not about exotic low-level tricks. In most production services, the biggest wins come from finding work the application is doing multiple times when once would suffice, or work it does not need to do at all. The tools to find these problems (flame graphs, continuous profilers) are well established. The barrier is that most teams have never actually looked.