Kubernetes Pod Pending: Causes, Diagnosis, and Fixes

A pod in Pending is not an application failure yet. Kubernetes has accepted the pod, but it has not completed the work required to run it—most often, the scheduler has not assigned a node. I start by separating that state from Running but NotReady: the first is a placement or setup investigation; the second is usually a container, probe, or dependency investigation.

The scheduler is usually willing to tell you exactly why it declined the pod. Read its event before changing YAML. A message such as 0/4 nodes are available: 2 Insufficient cpu, 2 node(s) didn't match Pod's node affinity/selector is not generic noise; it is the list of filters that eliminated the candidate nodes.

Start with the pod’s actual scheduling evidence

Pending is a pod phase, not a diagnosis. The API server may be waiting for a schedulable node, an unbound volume, or another prerequisite. Kubernetes documents the phase and its transitions in the pod lifecycle documentation.

I run these commands in this order because they establish whether I am looking at a scheduler problem at all:

Confirm the phase and node assignment
```
kubectl get pod <name> -n <namespace>
kubectl get pod <name> -n <namespace> -o jsonpath='{.status.phase}'
```
A Pending pod normally has no node in the NODE column. If the phase is Running but the READY column is short of its expected count, stop here: scheduling has succeeded. Check the pod’s conditions and Events, then inspect container logs and the readiness-probe path, port, delays, and timeouts instead.
```
kubectl get pod <pod-name> -n <namespace> -o wide
kubectl logs <pod-name>
kubectl get pods -A
```
Read the scheduler event
```
kubectl describe pod <name> -n <namespace>
```
Scroll to Events. For a true scheduling problem, I expect one or more FailedScheduling events near the bottom. Read the whole message, including the count in front of each reason. “0/4 nodes are available” tells you that Kubernetes evaluated four nodes; the rest of the message tells you why none passed. The kubectl reference is useful when you need to adjust the inspection command, but the event itself should drive the next action.
Compare the current and earlier events
```
kubectl get events -n <namespace> --field-selector involvedObject.name=<name>
kubectl get events -n <namespace> --field-selector involvedObject.name=<name> --sort-by=.lastTimestamp
```
Keep the sequence, not just the most recent line. A pod can begin by waiting for a PVC, then become unschedulable because the bound volume is restricted to a zone. Repeated events also show whether autoscaling or a recent rollout changed the set of eligible nodes.

Check resource requests against allocatable capacity

The scheduler places a pod according to its requests, not current CPU utilization on a dashboard. A node can look quiet and still lack the allocatable CPU, memory, ephemeral storage, or pod slots required by the new pod. Kubernetes describes that request-based model in its resource management guide.

For example, every replica of this pod requires two CPUs and 4 GiB of memory:

resources:
  requests:
    cpu: "2"
    memory: "4Gi"
  limits:
    cpu: "2"
    memory: "4Gi"

A node with 1.5 allocatable CPUs cannot take that pod, even when observed CPU usage is low. Confirm both the event and the allocatable values:

kubectl describe pod <pod-name> -n <namespace>
kubectl get nodes -o custom-columns=NAME:.metadata.name,CPU:.status.allocatable.cpu,MEMORY:.status.allocatable.memory

Look for Insufficient cpu or Insufficient memory in Events. The kube-scheduler documentation provides the broader placement model, but the operational detail is simpler: one eligible node must fit the request after Kubernetes reserves what it needs.

Do not confuse total cluster capacity with a schedulable slot

I often see a cluster with enough aggregate free memory but no node with enough contiguous allocatable memory for the next replica. That is fragmentation, usually caused by requests that are materially above observed steady-state use. Lowering a request just to clear a Pending event is not a fix if the workload genuinely needs that capacity; it trades an explicit scheduling refusal for contention later. Use observed workload behavior and an operating margin, then adjust the request deliberately.

Also check the node’s pod limit. CPU and memory can remain available after the kubelet has reached its maximum pod count:

kubectl describe node <node-name>
kubectl get pods -A --field-selector spec.nodeName=<node-name> --no-headers | wc -l

If you use Cluster Autoscaler, verify the node-group maximum and the shape of the available instance types. It can add a node for a resource shortage only when a permitted node group can satisfy the pod’s requests and placement rules. It cannot make an oversized request, an exhausted pod limit, or an incompatible affinity rule valid.

Trace the placement rules that eliminate nodes

Once capacity is plausible, inspect every hard constraint in the pod template: nodeSelector, required node affinity, pod affinity and anti-affinity, taints and tolerations, topology spread, and node readiness. Kubernetes applies these together, not one at a time. A node that has sufficient memory can still be ineligible because a selector, taint, or topology rule rejects it. The node assignment documentation covers these placement controls.

Start with the lowest-complexity check. A selector such as nodeSelector: { workload: batch } needs an exact key-value match. A renamed label or a node-pool change is enough to strand every replica.

kubectl get nodes --show-labels
kubectl describe node <node-name>

How Kubernetes placement constraints affect scheduling

Constraint	What it matches	Typical fix
Node selector	Exact key-value labels on a node	Correct the selector or apply the intended label to eligible nodes
Node affinity	Required or preferred node label expressions	Review operators, values, and whether a required rule is unnecessarily strict
Pod anti-affinity	Existing pods, labels, and topology domains	Relax the rule, change the topology key, or add capacity in another domain
Taint and toleration	A node taint against a matching pod toleration	Add a narrowly scoped toleration, or remove or correct the taint

Audit hard affinity and anti-affinity before relaxing them

Required node affinity and nodeSelector are hard filters; preferred affinity influences placement but does not by itself exclude a node. Review each matchExpressions operator and value against the labels that are actually on the current nodes. An obsolete pool label in a required rule is functionally identical to a selector mismatch.

Required pod anti-affinity is another common trap. It is useful when replicas must not share a topology domain, but it can eliminate every node when the cluster has too few zones or hostnames, or when the selector matches more existing pods than intended. Compare the rule’s namespace scope, label selector, topologyKey, and required-versus-preferred policy with the running pods. Do you really need a hard rule at hostname granularity, or is zone-level separation the actual availability requirement?

Match taints with the narrowest toleration that works

A taint repels a pod unless its toleration matches the taint’s key, effect, and, when specified, value. Scheduler events commonly say had untolerated taint. Inspect the node and the rendered pod spec, then decide whether the workload is actually intended to run in that isolated pool.

kubectl describe node <node-name>
kubectl get pod <pod-name> -n <namespace> -o yaml

Do not add a catch-all toleration to make the event disappear. It can place a workload on nodes reserved for system services, GPUs, or a different tenancy boundary. The taints and tolerations reference explains the matching rules.

Investigate storage before blaming the scheduler

A PVC can keep a pod Pending before ordinary node placement begins. Kubernetes may need to bind the claim to a compatible PersistentVolume, or dynamically provision one, before it can settle on a node. A brief wait is normal; a persistent wait means the claim’s requested capacity, access mode, StorageClass, or provisioning path does not match what the cluster can provide. See the Kubernetes documentation for PersistentVolumes and the pod lifecycle.

Inspect the claim and its provisioning events

kubectl get pvc -n <namespace>
kubectl describe pvc <claim-name> -n <namespace>
kubectl get pv
kubectl get storageclass

A claim in Pending has not bound. Its Events will usually distinguish a missing StorageClass, a failed provisioner request, an incompatible access mode, and insufficient backing capacity. Fix that condition first. Deleting and recreating the pod does not repair a provisioner or manufacture a compatible volume.

Check zone and volume topology after the claim binds

A bound volume can still narrow placement. In a multi-zone cluster, the PV may be accessible only from its provisioned zone, or its node-affinity rules may exclude nodes that otherwise pass CPU and memory checks. Compare the PV topology with the labels on eligible nodes and re-read the pod Events after binding. This is a topology constraint, not evidence that the cluster is generally out of capacity. Kubernetes documents the related scheduling behavior in its topology spread constraints guide.

Account for namespace policy, topology spread, and priority

Available cluster capacity does not guarantee that a particular namespace can consume it. In multi-tenant clusters, quotas, topology policy, priority, and node pressure deliberately restrict the scheduler’s options.

Check ResourceQuota before adding nodes

A ResourceQuota caps the resources and object counts a namespace may consume. If the namespace has hit its ceiling, adding worker nodes does not help. Compare the quota with the pod’s requests and the admission or scheduling events:

kubectl get resourcequota -n payments
kubectl describe resourcequota -n payments
kubectl describe pod checkout-7d9f -n payments

The safe correction may be to remove an unused workload, correct an inflated request, or have the quota owner raise the budget. Quotas are often the only control preventing one namespace from consuming a shared node pool, so treating them as an obstacle to delete is a poor default.

Review strict topology spread and priority behavior

Topology spread constraints balance replicas across zones, regions, or hostnames. Verify that the referenced node labels exist and inspect topologySpreadConstraints.whenUnsatisfiable. A hard setting can leave a pod Pending when the only remaining placement would break the requested distribution; a soft setting permits placement when perfect balance is impossible. Consider storage topology and affinity at the same time, because their combined candidate set—not each field in isolation—is what the scheduler sees.

Priority changes who gets scarce capacity, not whether a feasible placement exists. Higher-priority pods may preempt lower-priority workloads, which can leave those lower-priority pods Pending. Finally, use kubectl describe node to look for memory, disk, or PID pressure and node readiness. A node under pressure or marked NotReady is not a reliable scheduling target, regardless of what a utilization graph suggests.

Close the loop after the change

After changing capacity, labels, a toleration, storage, or quota, watch the same evidence that identified the fault. The new event should disappear or be replaced by a concrete next constraint; the pod should gain a node; and only then should application-level readiness become the active investigation.

kubectl get pod <name> -n <namespace> -w
kubectl describe pod <name> -n <namespace>

The practical rule is simple: diagnose a Pending pod from its latest FailedScheduling or PVC event, then change the single constraint that event proves is blocking placement—never start by loosening every scheduling rule at once.