Kubernetes Fault Domains: What Default Really Means
No, Kubernetes is not a fault domain by default. A Kubernetes cluster is a collection of machines, and unless you explicitly configure it to spread workloads across distinct failure zones, the scheduler will happily place replicas on the same rack, same power supply, or even the same physical host. This article explains what fault domains are, why Kubernetes doesn’t enforce them out of the box, and how to actually implement them.
What Is a Fault Domain?
A fault domain is a set of components that share a single point of failure. If one part of that domain fails, everything inside it goes down together. Common examples include a physical server, a rack with a shared power distribution unit, a network switch, or an entire availability zone in a cloud provider. The goal of spreading workloads across fault domains is to ensure that a single failure doesn’t take out all copies of your application.
In on-premises data centers, fault domains are often defined by hardware boundaries: a server, a rack, a row, or a data hall. In the cloud, they map to availability zones (AZs) or, in some cases, to individual hosts within a zone. The key point is that a fault domain is a logical grouping based on shared risk, not a Kubernetes concept by itself.
How Kubernetes Handles (or Ignores) Fault Domains
Kubernetes has built-in awareness of certain topology labels, but it does not automatically use them to spread pods. The scheduler’s default behavior is to place pods on nodes that have enough resources, regardless of whether those nodes are in the same rack or different racks. This means that if you create a Deployment with three replicas, they might all land on the same node, or on three nodes in the same rack, unless you tell the scheduler otherwise.
Kubernetes does recognize standard labels like topology.kubernetes.io/zone and topology.kubernetes.io/region, which cloud providers set automatically. However, these labels are just metadata—they don’t influence scheduling unless you use them in a podAntiAffinity rule or a topologySpreadConstraints configuration. Without those, the scheduler treats all nodes as equally valid, and it will happily place all replicas in a single zone if that’s where the resources are.
Why the Default Isn’t Fault-Tolerant
The default behavior exists for a reason: simplicity and efficiency. The scheduler’s primary job is to find nodes that can run the pod, not to optimize for resilience. If you don’t specify any constraints, it will pack pods densely to use resources efficiently. This is fine for development environments or applications that don’t need high availability, but it’s a problem for production workloads that expect to survive a node or zone failure.
Consider a typical three-node cluster. If you deploy a web application with three replicas and no anti-affinity rules, the scheduler might place all three on the same node if that node has enough CPU and memory. That node fails, and your entire application goes down. Even if the scheduler spreads them across different nodes, those nodes might be in the same rack, so a power failure in that rack takes out the whole app.
How to Actually Use Fault Domains in Kubernetes
To make Kubernetes respect fault domains, you need to explicitly tell it to. There are two main mechanisms: podAntiAffinity and topologySpreadConstraints.
- podAntiAffinity – This rule says “don’t place this pod on the same node (or zone) as another pod with a certain label.” For example, you can require that no two replicas of your app run in the same availability zone.
- topologySpreadConstraints – This gives you more fine-grained control. You can specify that pods should be spread evenly across zones, or across nodes, or both. You can also set a
maxSkewto allow some imbalance if needed.
Here’s an example of a topology spread constraint that forces even distribution across zones:
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app: my-app
This tells the scheduler to spread pods across zones, with at most a difference of one pod between any two zones. If a zone is unavailable, the scheduler will still place pods, but it will try to keep the distribution even.
Common Mistakes and Misconceptions
One common mistake is assuming that a ReplicaSet or Deployment automatically spreads replicas across nodes. It doesn’t—the scheduler will place them wherever it sees fit, and if you have a single node, all replicas will run on that node. Another mistake is using podAntiAffinity with requiredDuringScheduling on a small cluster. If you have three replicas and only two nodes, the third replica will be stuck in Pending because it can’t find a node that satisfies the anti-affinity rule.
Another misconception is that fault domains are only relevant in the cloud. On-premises clusters also have physical boundaries—racks, power circuits, network switches—and you can label nodes accordingly. For example, you can add a label like rack=rack1 to each node and then use that label in a topology spread constraint. This is a common practice in bare-metal Kubernetes deployments.
Cost and Trade-offs of Spreading Workloads
Spreading workloads across fault domains has costs. First, it may increase latency if pods in different zones need to communicate over the network instead of over a local switch. Second, it can reduce resource utilization because you might have to leave some capacity idle in each zone to handle failover. Third, it complicates operations—you need to monitor which zone each pod is in, and you might need to handle cross-zone traffic costs in the cloud.
However, for critical applications, the cost of downtime is usually much higher than these trade-offs. The key is to decide what level of availability you need. If you can tolerate a few minutes of downtime, you might not need strict zone spreading. If you need 99.99% availability, you should spread across multiple zones and possibly multiple regions.
Alternatives to Built-in Fault Domains
Kubernetes itself doesn’t provide a full fault domain management system, but there are tools that build on it. For example, Karpenter (a node autoscaler) can be configured to spread nodes across zones and instance types. Descheduler can help rebalance pods after the fact. Some managed Kubernetes services, like Amazon EKS and Google GKE, have their own node pool and zone spread features. But these are all add-ons—the core Kubernetes scheduler still needs explicit instructions.
Another alternative is to use a service mesh or an external load balancer that can route traffic to healthy pods, but that doesn’t solve the placement problem. The fundamental issue is that Kubernetes won’t spread your pods unless you ask it to.
Frequently Asked Questions
Does Kubernetes automatically spread pods across nodes?
No. By default, the scheduler places pods on any node with sufficient resources. It does not consider node or zone diversity unless you add constraints like podAntiAffinity or topologySpreadConstraints.
What is the difference between podAntiAffinity and topologySpreadConstraints?
podAntiAffinity prevents pods from being placed on the same topology (e.g., node or zone) as another pod. topologySpreadConstraints aims to distribute pods evenly across a topology, allowing a controlled skew. Anti-affinity is binary (yes/no), while spread constraints are more flexible.
Can I use fault domains in a single-node cluster?
No. A single node is one fault domain. If that node fails, everything fails. To use fault domains, you need at least two nodes in different domains (e.g., different racks or zones).
Do cloud providers set topology labels automatically?
Yes. Most managed Kubernetes services automatically label nodes with topology.kubernetes.io/zone and topology.kubernetes.io/region. You can use these labels in your scheduling constraints without manual setup.
What happens if a zone fails when I have spread constraints?
If a zone goes down, the pods in that zone become unavailable. The scheduler will try to place new pods in the remaining zones, but if you have a requiredDuringScheduling constraint, it may not place them if the distribution would be too skewed. Using preferredDuringScheduling allows the scheduler to place pods even if the distribution is uneven.
Related
Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.