We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design Masterclass · Kubernetes Containers
Free preview: the first 2 parts of this 6-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design courseread the rest of this lesson in
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design coursein Kubernetes Containers
This lesson is part of the full System Design Masterclass. Unlock it along with 769 other lessons for a one-time payment.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime Access18 Foundation lessons are free, no payment needed. Try them now
It is easy and good
“It is really nice and good Please join the course”
A Deployment says "run 5 replicas somewhere in the cluster." But some workloads don't care about replicas. They need coverage of every node. A log collector like Fluentd needs to tail container logs from every node's filesystem. A monitoring agent like the Datadog agent needs to report CPU, memory, and disk metrics from every machine. A CNI plugin like Calico needs to configure pod networking on every node, because without it, pods on that node cannot communicate at all.
You cannot fake this with a Deployment. If you set replicas to match the node count, the scheduler might pack two pods onto one beefy node and leave a small node with zero. Add a node at 3 AM via the autoscaler and nobody bumps the replica count. Remove a node and one pod is orphaned.
DaemonSets solve this automatically. They guarantee exactly one pod per qualifying node. When a new node joins the cluster, the DaemonSet controller creates a pod on it within seconds. When a node is removed, the pod is garbage collected. You never think about replica counts. Datadog's own documentation recommends DaemonSets as the only correct deployment model for their agent, and Cilium's install guide deploys it as a DaemonSet because there is no alternative that ensures per-node coverage.
| Use Case | DaemonSet Pod | Why Per-Node |
|---|---|---|
| Log collection | Fluentd, Filebeat, Fluent Bit | Reads container logs from node filesystem (/var/log/containers) |
| Monitoring | Datadog, Node Exporter | Reports node-level CPU, memory, disk, network |
| Networking | Calico, Cilium, AWS VPC CNI | Configures pod networking on each node |
| Storage | CSI drivers (EBS, EFS, Ceph) | Manages volume mounts on each node |
| Security | Falco, Aqua | Monitors system calls on each node |
Node selection: Use nodeSelector or affinity to limit which nodes get the DaemonSet pod. Run GPU monitoring only on GPU nodes. Run storage drivers only on nodes with attached disks. Skip small spot instances that should not carry the overhead.
Tolerations: DaemonSet pods often need to run on control plane nodes (tainted with node-role.kubernetes.io/master:NoSchedule) or dedicated nodes that regular pods avoid. Add tolerations to your DaemonSet spec so the pod can schedule on those nodes.
Update strategy: DaemonSets support RollingUpdate (one node at a time) and OnDelete (manual). For CNI plugins, rolling updates are mandatory. Updating Calico on all nodes simultaneously kills pod-to-pod networking cluster-wide. The default maxUnavailable: 1 means exactly one node is updating at any moment.
Resource budgets: DaemonSet pods run on every node, including small ones. If your Fluentd pod requests 512 MB of RAM, that 512 MB is consumed on every node in the cluster. A 100-node cluster uses 50 GB of RAM just for log collection. Pick resource requests carefully.