Service Mesh

When you split an application into dozens or hundreds of microservices, every feature now depends on many network calls between services. Each of those calls needs the same boring but important things: encryption so traffic cannot be sniffed, retries when a call fails, timeouts so a slow service does not hang everyone, and metrics so you can see what is happening. Writing that code in every service, in every language, is repetitive and error prone.

A service mesh moves all of that into the infrastructure. It is made of two parts. The data plane is a fleet of small proxies, one running alongside each service instance, that intercept all inbound and outbound traffic. The control plane is the brain that configures those proxies, distributes certificates, and collects telemetry. Popular implementations include Istio and Linkerd, both of which run on Kubernetes.

The key idea is that your application code stays simple. A service just makes a normal HTTP or gRPC call to another service by name. The proxy next to it transparently encrypts the connection, picks a healthy backend, retries on failure, and records how long it took.

The most common pattern is the sidecar proxy. In Kubernetes, the mesh injects an extra container, usually Envoy or Linkerd's own Rust proxy, into each pod. Then it rewrites the pod's network rules with iptables so that every packet leaving or entering the application container is routed through that proxy first. The application has no idea this is happening.

Because the proxy sees every request, it can do a lot. It terminates and originates mutual TLS, so service A and service B both present certificates and verify each other before any data flows. It load balances across the healthy instances of the target service. It applies policies the control plane pushed down, such as retry budgets, circuit breaking, and traffic splitting for canary releases. It emits a metric for every single call, which is where the famous golden signals of latency, traffic, errors, and saturation come from.

The control plane watches the cluster's service registry, so when new instances start or old ones die, it updates the proxies within seconds. It also acts as a certificate authority, issuing short-lived certificates to each workload and rotating them automatically. A newer approach, used by Istio's ambient mode and by Cilium, drops the per-pod sidecar in favor of a shared per-node proxy to cut memory and latency overhead.

A service mesh earns its keep when you have many services, multiple teams, and real requirements around zero-trust security, traffic control, and observability. If you need mutual TLS between every service for compliance, or you want to do canary deployments by shifting 5 percent of traffic to a new version, the mesh gives you that with configuration instead of code.

The cost is real. Every request now passes through two extra proxies, which adds latency, often around 1 to 3 milliseconds per hop, and consumes CPU and memory across the cluster. The control plane is another distributed system you have to run, upgrade, and debug, and mesh failures can be confusing because the problem lives in the network layer, not your code.

For a handful of services, a service mesh is usually overkill. A simpler API gateway, a client-side library, or built-in Kubernetes networking will do the job with far less operational weight. Teams often adopt a mesh only after they have felt the pain of inconsistent retries, missing metrics, or unencrypted internal traffic across a large fleet.

Imagine a payments platform with a checkout service that calls a fraud-detection service, which in turn calls an account service. Without a mesh, each team writes its own retry and timeout logic, some forget to add it, and a slow fraud service can stall every checkout.

With Istio installed, an operator writes a short configuration that says fraud-detection calls should time out after 500 milliseconds, retry twice, and trip a circuit breaker if more than half of recent calls fail. They also enable strict mutual TLS for the whole namespace. No application team changes a line of code.

Now when the fraud service has a bad deploy, the proxies stop sending traffic to the broken instances, retry against healthy ones, and the dashboard immediately shows the error spike with exact latency percentiles. The team rolls out the fix as a canary, shifting 10 percent of traffic to the new version, watching the metrics, then ramping to 100 percent, all controlled by the mesh.