We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design Masterclass · Observability Monitoring
Free preview: the first 2 parts of this 9-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design courseread the rest of this lesson in
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design coursein Observability Monitoring
This lesson is part of the full System Design Masterclass. Unlock it along with 769 other lessons for a one-time payment.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime Access18 Foundation lessons are free, no payment needed. Try them now
It is easy and good
“It is really nice and good Please join the course”
Brendan Gregg, a performance engineer at Netflix (and Intel, and Sun Microsystems, and the creator of flame graphs), developed the USE method because engineers kept spending days debugging performance problems that could be found in minutes with a systematic approach.
His observation: most performance problems are caused by a resource bottleneck, some component (CPU, memory, disk, network, connection pool) is being overused. The USE method provides a checklist: for every resource, check three things. Utilization, Saturation, and Errors. If any of those are abnormal, you've found your bottleneck. If none are, the problem isn't resource-related.
Where RED monitors services from the user's perspective, USE monitors infrastructure from the resource's perspective. Together, they cover both sides: "Is the service behaving correctly?" (RED) and "Does the infrastructure have the capacity to support it?" (USE).
For every hardware or software resource, check:
| Signal | Definition | Example |
|---|---|---|
| Utilization | Time the resource was busy or % of capacity used | CPU at 72%, memory at 85%, disk at 90% |
| Saturation | Extra work the resource can't service, usually queued | 20 requests waiting for CPU, 5 threads waiting for a lock |
| Errors | Count of error events from the resource | ECC memory errors, disk read errors, NIC CRC errors |
Apply USE to every resource:
| Resource | Utilization | Saturation | Errors |
|---|---|---|---|
| CPU | % time busy | Run queue length, load average | Machine check exceptions |
| Memory | % used (available vs total) | Swap usage, OOM kills | ECC errors |
| Disk | % time busy, % capacity | I/O wait queue length | Disk read/write errors |
| Network | utilization | Packets dropped, retransmits | CRC errors, link errors |
| Connection pool | Active/total connections | Connections waiting | Connection refused errors |
| Thread pool | Active/total threads | Tasks queued | Thread exhaustion errors |
The method is systematic: go through each resource, check all three signals. High utilization means the resource is busy, it might become a bottleneck soon. High saturation means it already is a bottleneck, work is queuing up. Errors mean the resource is malfunctioning.