APM
Application Performance Monitoring: tools that track request latency, error rates, and dependencies in real time. Datadog, New Relic, and Grafana are popular APM platforms.
What is APM?
In short
APM (Application Performance Monitoring) is a category of tooling that measures how your running application behaves in production by tracking request latency, error rates, throughput, and the chain of internal and external dependencies each request touches. It answers "why is this endpoint slow or failing right now" by tracing a single request across services, databases, and queues, then aggregating those traces into dashboards and alerts.
What APM Actually Measures
APM is the part of observability focused on the performance and health of application code, not the underlying servers. Infrastructure monitoring tells you a CPU is at 90 percent. APM tells you that the /checkout endpoint's p99 latency jumped from 180ms to 2.4 seconds because a downstream Postgres query started doing a full table scan.
The four numbers almost every APM tool centers on are latency (how long requests take, reported as p50, p95, and p99 percentiles, not just averages), error rate (the share of requests returning 5xx or throwing exceptions), throughput (requests per second or per minute), and saturation (how close a resource like a connection pool or thread pool is to its limit). Google's SRE book calls latency, traffic, errors, and saturation the four golden signals, and APM dashboards are built around them.
The reason percentiles matter more than averages: if 99 out of 100 requests take 50ms and one takes 5 seconds, the average is a misleading 100ms, but your p99 is 5 seconds, and that one slow request is the customer who abandons the cart. APM tools surface the tail, not the mean.
How It Works Under the Hood
An APM agent or SDK runs inside your application process. It hooks into common libraries (HTTP frameworks, database drivers, HTTP clients, message queue clients) using instrumentation, then records a span for each unit of work: an incoming HTTP request, a SQL query, a call to Redis, a publish to Kafka. Each span carries a start time, a duration, and metadata like the SQL statement or the URL.
Spans from a single request are stitched together into a trace using a shared trace ID that is propagated across service boundaries, usually through HTTP headers. This is what lets you click one slow request in the dashboard and see the full waterfall: 40ms in your service, 1.8 seconds waiting on a database, 200ms in a third-party payment API. Distributed tracing is the backbone of APM in a microservices setup.
Because tracing every request at scale is expensive, most APM systems sample. Head-based sampling decides at the start of a request whether to keep it (for example, keep 10 percent). Tail-based sampling buffers traces and keeps the interesting ones, like every trace that errored or exceeded a latency threshold, which costs more to run but loses fewer of the traces you actually care about.
The modern standard for the instrumentation layer is OpenTelemetry, a vendor-neutral set of SDKs and a wire format. You instrument once with OpenTelemetry and can ship the data to Datadog, Grafana, New Relic, or your own backend, which avoids locking your code to one vendor's proprietary agent.
When To Use It and the Trade-Offs
Reach for APM the moment you have more than one service or any meaningful traffic. Logs tell you what happened in one place, metrics tell you aggregate trends, but only tracing tells you the causal path of a single slow or failed request across many places. For debugging production incidents and finding the one slow dependency in a chain of ten, nothing else gives you that view as fast.
The costs are real. Agents add CPU and memory overhead, typically a few percent, and high-cardinality data plus full-fidelity traces get expensive fast. Datadog and New Relic bill on hosts, ingested data volume, and indexed spans, and teams routinely get surprised by bills that scale with traffic. Sampling and dropping low-value spans are how you keep the bill sane, at the cost of not capturing every request.
There is also instrumentation effort and a privacy risk: spans can accidentally capture full SQL with values, request bodies, or tokens, so you need scrubbing rules. APM is not a replacement for logs and metrics either. The healthy pattern is the three pillars of observability working together: metrics for alerting and trends, logs for detailed context, traces for following a request end to end.
A Concrete Example
Picture an e-commerce checkout that suddenly gets slow at 8pm. The on-call engineer opens the APM service map and sees that the checkout service's p99 latency is 3 seconds, up from 200ms, while throughput is normal. The error rate is flat, so requests are succeeding but crawling.
Clicking a slow trace reveals the waterfall. The checkout service itself spends 30ms. Then there is a 2.7 second span on a single SQL query against the orders table. The span metadata shows the query, and a quick check confirms an index was dropped in last night's migration, forcing a sequential scan that got slow as the table grew during peak traffic.
Without APM, that investigation is grepping logs across several services and guessing. With it, the path from alert to root cause is a few clicks and a couple of minutes, because the tool already recorded exactly where the time went on the request that mattered.
Where it is used in production
Datadog APM
Auto-instruments apps in many languages, builds live service maps, and correlates traces with logs and infrastructure metrics in one product.
New Relic
One of the earliest APM vendors; agents track transaction traces, error analytics, and database query performance, billed on ingested data and users.
Grafana (Tempo plus Prometheus)
Open-source observability stack where Tempo stores traces, Prometheus stores metrics, and Grafana visualizes them together, often fed by OpenTelemetry.
OpenTelemetry
Vendor-neutral CNCF standard for generating traces and metrics; not a backend itself, but the instrumentation layer most modern APM setups now build on.
Frequently asked questions
- What is the difference between APM and observability?
- APM is a focused product category about application performance, centered on traces, latency, and errors. Observability is the broader practice of being able to understand any system state from its outputs, built on three pillars: metrics, logs, and traces. APM mostly covers the traces pillar plus some metrics, so it is a major part of observability but not the whole thing.
- What is the difference between APM and logging?
- Logs are discrete text records of events in one component, good for detailed context like a stack trace. APM follows a single request across many components and tells you where the time and failures occurred. Logs answer what happened here; APM answers where in the whole request path the problem is. You want both, often linked by a shared trace ID.
- Why do APM tools focus on p99 instead of average latency?
- Averages hide the slow tail. If most requests are fast but one in a hundred takes seconds, the average looks fine while real users hit the slow path. The p99 latency is the value 99 percent of requests come in under, so it exposes the worst common experience, which is usually what causes abandoned carts and angry users.
- Does running an APM agent slow my application down?
- There is some overhead, typically a few percent of CPU and memory, because the agent intercepts library calls and records spans. For most applications that is an acceptable trade for the visibility. You control the cost with sampling, so you trace a fraction of requests rather than all of them, while still keeping every errored or slow trace.
- Is OpenTelemetry an APM tool?
- No. OpenTelemetry is the instrumentation standard that generates traces and metrics in a vendor-neutral format. It does not store or visualize the data. You pair it with a backend like Datadog, Grafana Tempo, or New Relic, which does the storage, dashboards, and alerting. The benefit is you instrument once and can switch backends without rewriting your code.
Learn APM hands-on
This page explains the idea. The full lesson lets you step through the ring as servers join and leave, read the implementation, and check yourself with a quiz. It is one of 760+ lessons in the System Design Masterclass, from your first API call to distributed consensus. Eleven Foundation lessons are free, no signup. Lifetime access is ₹499 in India or $7.99 worldwide, one payment, no subscription.
Related lessons
Lessons that touch on APM as part of a larger topic.
See also
Related glossary terms you might want to look up next.
Distributed Tracing
Tracking a request as it flows through multiple services in a distributed system. Each service adds its trace, creating a full picture of the request journey.
Metrics
Numerical measurements collected over time that describe system behavior: request rate, error rate, latency percentiles, CPU utilization. Prometheus is the standard collector.
Observability
The ability to understand a system's internal state from its external outputs. Built on three pillars: metrics, logs, and traces.
Health Check
An endpoint or mechanism that reports whether a service is running and healthy. Load balancers use health checks to route traffic away from unhealthy instances.
Logging
Recording discrete events with timestamps, severity levels, and context. Structured logs (JSON) are searchable; unstructured logs (plaintext) are not. Ship them to a central system.
Alerting
Automatically notifying engineers when metrics cross predefined thresholds. Good alerts are actionable, not noisy. PagerDuty and Opsgenie route alerts to the right on-call person.