Your microservices architecture has 30 services. Every service needs:

• Mutual TLS (encrypted service-to-service communication)
• Retries with exponential backoff (resilience)
• Circuit breakers (fault tolerance)
• Load balancing (traffic distribution)
• Distributed tracing (observability)
• Rate limiting (protection)
• Canary deployments (safe rollouts)

You could implement these in every service. But that means every service, whether written in Go, Python, Java, or Node.js, needs the same networking logic. Every team maintains the same retry policies, the same TLS configuration, the same circuit breaker thresholds. When you need to change the retry policy, you update 30 services.

Netflix tried the library approach first. They built Ribbon (load balancing), Hystrix (circuit breaking), and Eureka (service discovery) as Java libraries. It worked, but only for Java services. When teams started writing Go and Python services, they needed to reimplement everything. Maintaining parity across three language implementations became a full-time job.

A service mesh moves all of this into the infrastructure layer. Your services handle business logic. The mesh handles networking. Your Go service and your Python service automatically get mTLS, retries, tracing, and load balancing without a single line of networking code.

A service mesh has two components:

Data Plane: the sidecar proxies. Every service instance gets a lightweight proxy (typically Envoy) deployed alongside it. All traffic in and out of the service flows through this proxy. The proxy handles encryption, retries, circuit breaking, load balancing, and telemetry collection, transparently.

Control Plane: the brain. A centralized component that configures all the proxies. You tell the control plane "enable mTLS between all services" or "route 5% of traffic to the canary version" and it pushes this configuration to every proxy in the mesh.

The full mTLS handshake as a sequence of events. Service A sends plaintext to its local proxy. Proxy A initiates TLS with Proxy B, presenting its mesh-issued certificate. Proxy B verifies and presents its own cert. Both sides authenticated, traffic encrypted with AES-256-GCM. Neither application touches TLS. Certificates rotate automatically every 24 hours.

The mesh splits traffic between service versions. 95% goes to stable v1, 5% to canary v2. Flagger monitors error rates and latency from Prometheus. If v2 looks healthy, traffic gradually shifts to 100%. If errors spike, all traffic routes back to v1 instantly. Zero-downtime deployments with automatic rollback.

Traffic Management:

• Canary deployments: route 5% of traffic to v2, 95% to v1. Gradually shift if metrics look good.
• A/B testing: route users with header x-test-group: B to the experimental service.
• Fault injection: artificially add 500ms latency or 5% error rate to test resilience. Netflix uses this in production through their Chaos Engineering program.
• Traffic mirroring: send a copy of production traffic to a new version without affecting users. Test with real traffic patterns before any user sees the new code.

Security:

• Mutual TLS: every service-to-service call is encrypted and authenticated, automatically. HIPAA, SOC2, and PCI-DSS compliance teams love this.
• Authorization policies: "Service A can call Service B, but Service C cannot." Zero-trust networking at the service level.
• Certificate rotation: automatic, no service restarts needed. Istio rotates every 24 hours by default.

Observability:

• Distributed tracing: every request gets a trace ID that follows it through the entire call chain. Works with Jaeger, Zipkin, and Datadog without instrumenting application code.
• Metrics: request rate, error rate, latency percentiles (p50, p95, p99), for every service, automatically. Exported to Prometheus.
• Access logging: who called whom, when, with what result. Audit trails for free.

Three dominant options with fundamentally different trade-offs. Istio: most features, highest resource cost (~100MB RAM per sidecar). Linkerd: lightweight and simple (~20MB RAM per sidecar). Cilium: eBPF-based, kernel-level networking (<1ms , no per-pod memory overhead). Pick based on your team size, feature requirements, and tolerance for operational complexity.

When to use a service mesh:

• You have 20+ services and cross-cutting concerns (mTLS, retries, tracing) are painful to implement per-service.
• Multiple programming languages. A mesh handles networking uniformly regardless of language.
• You need fine-grained traffic control (canary, A/B testing, fault injection).
• Security compliance requires mTLS everywhere.

When NOT to use a service mesh:

• You have fewer than 10 services. The operational overhead of the mesh exceeds its benefits. Implement retries and tracing with a shared library instead.
• Your team does not have Kubernetes expertise. Service meshes are complex to operate and debug.
• Latency is ultra-critical. Every request goes through two extra proxy hops (sender-side and receiver-side), adding 1-3ms per hop. For a service making 20 internal calls per request, that is 40-120ms of added latency.

The operational cost is real. The mesh itself needs monitoring, upgrading, and debugging. When requests fail, is it the application or the proxy? Mesh version upgrades require careful rollout across all proxies. Memory overhead: each Envoy sidecar uses 50-100MB RAM per instance. At 200 pods, that is 10-20GB of RAM just for proxies.

Lyft (where Envoy was born) publishes detailed metrics on mesh overhead. At their scale, the proxy latency is 0.2ms per hop on average. But the observability, mTLS, and traffic management they get in return saves hundreds of engineering hours per month.