We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design and AI Engineering, by Roni Das. 984 interactive lessons.
Start LearningIntermediate · Microservices Architecture
A dedicated infrastructure layer for handling service-to-service communication in microservices. Manages load balancing, encryption, and observability automatically.
In short
A service mesh is a dedicated infrastructure layer that handles communication between microservices, moving concerns like mutual TLS encryption, retries, load balancing, and traffic metrics out of your application code and into a network of sidecar proxies that sit next to each service. The mesh gives every service-to-service call automatic security, reliability, and observability without developers writing any of that logic themselves.
In short
A service mesh is a dedicated infrastructure layer that handles communication between microservices, moving concerns like mutual TLS encryption, retries, load balancing, and traffic metrics out of your application code and into a network of sidecar proxies that sit next to each service. The mesh gives every service-to-service call automatic security, reliability, and observability without developers writing any of that logic themselves.
Learn it in the course
This page is the free part.
₹1,111 in India$25 everywhere elseonce, for the whole course
The System Design course covers Service Mesh across a run of lessons, not one page. These 3 alone are about 71 minutes of step-by-step reading, every one with a quiz.
Dedicated infrastructure for service-to-service communication, observability, security, and traffic management without changing application code.
A dedicated infrastructure layer for managing service-to-service communication, observability, security, and traffic control without changing application code.
Istio vs. Linkerd vs. Cilium, choosing and deploying a service mesh in practice.
all part of
System Design Masterclass
770 lessons · about 287 hours · 18 free to read
₹1,111in India, by UPI
$25everywhere else, by PayPal
It is really nice and good Please join the course
You own this course
Continue with Service Mesh. Conceptspay once,
yours for life
When you split an application into dozens or hundreds of microservices, every feature now depends on many network calls between services. Each of those calls needs the same boring but important things: encryption so traffic cannot be sniffed, retries when a call fails, timeouts so a slow service does not hang everyone, and metrics so you can see what is happening. Writing that code in every service, in every language, is repetitive and error prone.
A service mesh moves all of that into the infrastructure. It is made of two parts. The data plane is a fleet of small proxies, one running alongside each service instance, that intercept all inbound and outbound traffic. The control plane is the brain that configures those proxies, distributes certificates, and collects telemetry. Popular implementations include Istio and Linkerd, both of which run on Kubernetes.
The key idea is that your application code stays simple. A service just makes a normal HTTP or gRPC call to another service by name. The proxy next to it transparently encrypts the connection, picks a healthy backend, retries on failure, and records how long it took.
The most common pattern is the sidecar proxy. In Kubernetes, the mesh injects an extra container, usually Envoy or Linkerd's own Rust proxy, into each pod. Then it rewrites the pod's network rules with iptables so that every packet leaving or entering the application container is routed through that proxy first. The application has no idea this is happening.
Because the proxy sees every request, it can do a lot. It terminates and originates mutual TLS, so service A and service B both present certificates and verify each other before any data flows. It load balances across the healthy instances of the target service. It applies policies the control plane pushed down, such as retry budgets, circuit breaking, and traffic splitting for canary releases. It emits a metric for every single call, which is where the golden signals of latency, traffic, and errors come from.
The control plane watches the cluster's service registry, so when new instances start or old ones die, it updates the proxies within seconds. It also acts as a certificate authority, issuing short-lived certificates to each workload and rotating them automatically. A newer approach, used by Istio's ambient mode and by Cilium, drops the per-pod sidecar in favor of a shared per-node proxy to cut memory and latency overhead.
A service mesh earns its keep when you have many services, multiple teams, and real requirements around zero-trust security, traffic control, and observability. If you need mutual TLS between every service for compliance, or you want to do canary deployments by shifting 5 percent of traffic to a new version, the mesh gives you that with configuration instead of code.
The cost is real. Every request now passes through two extra proxies, which adds latency, often around 1 to 3 milliseconds per hop, and consumes CPU and memory across the cluster. The control plane is another distributed system you have to run, upgrade, and debug, and mesh failures can be confusing because the problem lives in the network layer, not your code.
For a handful of services, a service mesh is usually overkill. A simpler API gateway, a client-side library, or built-in Kubernetes networking will do the job with far less operational weight. Teams often adopt a mesh only after they have felt the pain of inconsistent retries, missing metrics, or unencrypted internal traffic across a large fleet.
Imagine a payments platform with a checkout service that calls a fraud-detection service, which in turn calls an account service. Without a mesh, each team writes its own retry and timeout logic, some forget to add it, and a slow fraud service can stall every checkout.
With Istio installed, an operator writes a short configuration that says fraud-detection calls should time out after 500 milliseconds, retry twice, and trip a circuit breaker if more than half of recent calls fail. They also enable strict mutual TLS for the whole namespace. No application team changes a line of code.
Now when the fraud service has a bad deploy, the proxies stop sending traffic to the broken instances, retry against healthy ones, and the dashboard immediately shows the error spike with exact latency percentiles. The team rolls out the fix as a canary, shifting 10 percent of traffic to the new version, watching the metrics, then ramping to 100 percent, all controlled by the mesh.
Istio
The most widely deployed open source mesh, using Envoy proxies and a control plane that handles mTLS, traffic splitting, and telemetry on Kubernetes.
Linkerd
A lightweight CNCF mesh with a purpose-built Rust micro-proxy, chosen by teams that want low latency and a smaller operational footprint than Istio.
Lyft
Built and open-sourced Envoy, the proxy at the heart of most meshes, to manage communication across its large microservices fleet.
AWS App Mesh
Amazon's managed service mesh that runs Envoy sidecars to control and monitor traffic across ECS, EKS, and EC2 workloads.
An API gateway sits at the edge and manages traffic coming from outside clients into your system, handling things like authentication, rate limiting, and routing. A service mesh manages traffic between your internal services. You often run both: the gateway handles north-south traffic, the mesh handles east-west traffic.
Most popular meshes like Istio and Linkerd are designed for Kubernetes and are easiest there, but some, such as Consul and AWS App Mesh, support virtual machines and non-Kubernetes workloads too. Kubernetes is the common case, not a hard requirement.
A sidecar proxy is a small proxy container deployed next to each service instance, usually one per pod. All traffic into and out of the service is routed through it, so it can transparently add encryption, retries, load balancing, and metrics without the service knowing.
Yes, slightly. Each request passes through extra proxies, typically adding 1 to 3 milliseconds of latency per hop plus some CPU and memory overhead. For most systems that is acceptable, but newer sidecar-less designs like Istio ambient mode and Cilium aim to reduce it.
It gives you consistent mutual TLS, retries, timeouts, circuit breaking, traffic splitting for canary releases, and uniform per-call metrics across every service, in any language, without changing application code. You can build pieces of this by hand, but the mesh makes it consistent and centrally controlled.
Microservices
An architecture where an application is split into small, independent services that communicate over the network. Each service owns its own data and can be deployed separately.
Service Discovery
The mechanism by which microservices find and communicate with each other. Services register themselves and others can look them up by name.
Kubernetes
An orchestration platform that automates deploying, scaling, and managing containerized applications. K8s is the operating system for your cloud.
Monolith
A single, unified application where all features share the same codebase and deployment. Simpler to start with but harder to scale individual parts.
Circuit Breaker
A pattern that stops calling a failing service after repeated failures, preventing cascade failures. Like an electrical circuit breaker that cuts power to prevent fires.
Bulkhead
A pattern that isolates different parts of a system so a failure in one part doesn't sink the whole ship. Named after the compartments in a ship's hull.
This page explains the idea. The "Service Mesh. Concepts" lesson builds it up step by step, with diagrams. It is part of the System Design Masterclass course, which has 770 lessons. 18 of them are free to read. You pay once and keep it for life, with no subscription.
Open the Service Mesh. Concepts lessoncourse 1
From absolute beginner to principal engineer, drawn step by step.
Asked about Service Mesh in an interview?
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (268 diagrams in all). A PDF you keep, junior to staff level.
India: UPI, cards or netbanking through Razorpay. Elsewhere: card or PayPal. Download at once from My packs.
