Kubernetes Node NotReady: Causes, Diagnosis, and Fixes

A Kubernetes node can remain reachable at the infrastructure layer while the control plane stops trusting it to run workloads. That distinction matters: a missed Kubelet heartbeat, exhausted disk, broken CNI path, or runtime failure can turn a healthy-looking server into a scheduling and availability risk.

For platform teams managing regulated or air-gapped environments, fast diagnosis is part of day-2 reliability and compliance, not just incident cleanup. Start by separating the status signal from the underlying failure, then trace the conditions and events that caused the node to leave the fleet.

Bring your Kubernetes fleet management into one control plane with Plural.

What Does Kubernetes Node NotReady Mean?

A Kubernetes node reports its health through a set of status conditions, including Ready. When Ready=True, the node is healthy and available to accept Pods. When Ready=False, Kubernetes considers the node unhealthy and stops scheduling new Pods there. This is the core meaning of a kubernetes node notready condition, but the status itself is a signal, not a diagnosis.

The Kubelet on each worker node periodically reports status to the Kubernetes API server. That status includes the node's conditions, addresses, capacity, allocatable resources, and information about the Kubelet and container runtime. A failure in that reporting path can therefore make a healthy-looking machine appear unavailable to the control plane. For a deeper explanation of the component responsible for these updates, see what the Kubernetes Kubelet does.

What happens to workloads on a NotReady node?

NotReady changes scheduling behavior immediately: new Pods are not placed on the affected node. Existing Pods may continue running while the node condition is being evaluated, so an alert does not necessarily mean that every workload has already stopped. The distinction matters during incident response. A node can be unable to accept new work while its existing containers remain reachable for some period.

If the condition persists, the controller manager evicts Pods from the node after the configured timeout, which defaults to five minutes according to the Kubernetes cluster debugging documentation. Eviction allows controllers such as Deployments and StatefulSets to recreate workloads elsewhere, assuming the cluster has sufficient capacity and the workload's scheduling constraints permit relocation. In a capacity-constrained fleet, that delay can turn a single node issue into a broader availability problem.

NotReady versus Unknown

NotReady and Unknown are related, but they describe different evidence. NotReady means Kubernetes has a negative health signal. Unknown means the node controller has not heard from the node during the configured node-monitor-grace-period, which defaults to about 50 seconds. At that point, the control plane cannot establish the node's current condition. A network partition, powered-off host, failed Kubelet, or API connectivity problem can all produce that uncertainty.

Start with the condition and its transition time, then inspect the node's events and Kubelet logs. The official references for Kubernetes node status and debugging a cluster define the status semantics and the resulting scheduling behavior. Treat the condition as an operational boundary: stop sending new work to the node, establish whether it can recover, and protect workloads before the eviction timeout is reached.

Why Does a Kubernetes Node Flip to NotReady?

A Kubernetes node becomes NotReady when the control plane can no longer establish that the node is healthy and available for scheduling. In most incidents, the status is a symptom of a failure in the node agent, the host, or the path between the node and the API server. The cause lists used by Dash0 and Groundcover follow the same practical pattern: start with Kubelet health. Then check resource pressure, networking, and the container runtime before treating the event as a scheduler problem.

Kubelet heartbeats stop

The Kubelet reports node status to the API server. If it crashes, hangs, is misconfigured, or loses access to the API server, its heartbeats stop and the node transitions away from Ready. A failed systemd unit, expired certificate, bad bootstrap configuration, or an overloaded host can all produce this result. Read the Kubelet's recent logs on the node, and use this Kubelet troubleshooting guide to connect log messages to the node condition.

Disk, memory, or PID pressure

Resource pressure is another common trigger. Kubernetes exposes DiskPressure, MemoryPressure, and PIDPressure as node conditions. Disk pressure can follow container image accumulation, full log partitions, or exhausted inode capacity. Memory pressure may result from host processes or workloads consuming available memory. PID pressure appears when the host approaches its process limit, preventing new processes from starting. The Kubernetes Node Status reference defines these pressure conditions and their relationship to node health: review the authoritative condition definitions.

Network and CNI failures

A healthy process cannot report status if the node cannot reach the API server. Broken routes, firewall changes, DNS failures, expired certificates, or a failed CNI plugin can interrupt kubelet-to-API communication. Check the node's route and DNS configuration, then inspect the CNI daemon and its interfaces. A network issue may also make workloads look unhealthy even when the host itself is running.

Runtime failures and planned maintenance

If containerd or CRI-O is stopped, unhealthy, or incompatible with the Kubelet, the node may fail readiness checks and lose the ability to manage containers. Check runtime service status and logs alongside the Kubelet logs. Finally, account for ordinary operational events: a node reboot, kernel update, autoscaling replacement, or Kubernetes upgrade can leave a node NotReady while services restart. Planned work should use cordon and drain first. An unexpected reboot requires checking boot logs, filesystem health, runtime startup, and whether the node rejoined with the expected labels and configuration.

These causes can overlap. For example, a runtime crash can fill a filesystem with stale logs, while a CNI failure can prevent both workload traffic and Kubelet heartbeats. Treat NotReady as a signal to correlate node conditions, service logs, and network state rather than as a single fault category.

How to Diagnose a NotReady Kubernetes Node with kubectl

Use a consistent sequence that moves from cluster-level confirmation to node-level evidence. This helps distinguish a missing heartbeat from resource pressure, a runtime failure, or a network problem before you restart or replace anything.

1. Confirm the node state and registration

   Start from the control plane with kubectl get nodes. This shows whether the node is registered and whether its status is NotReady. The command is the recommended first check when debugging cluster health. For a more useful view, include conditions and addresses:

   kubectl get nodes -o wide
   kubectl get node <name> -o yaml

   For a deeper explanation of the first command, see Plural's guide to kubectl get nodes.

2. Read the node conditions, status, and events

   Run kubectl describe node <name> and inspect the Conditions, Addresses, Allocatable, and Events sections. Focus on Ready, MemoryPressure, DiskPressure, PIDPressure, and NetworkUnavailable. A pressure condition points toward local capacity or process exhaustion. A Ready value of Unknown indicates that the node controller has not heard a heartbeat within the configured node-monitor-grace-period, which defaults to about 50 seconds in the Kubernetes node-status reference.

   The Kubernetes cluster debugging guide recommends kubectl describe node for detailed node conditions and events. The node status reference defines what each condition means.

3. Check registration, eviction, and scheduling events

   Query recent events to identify node registration failures, eviction activity, or transitions around the time the node became unhealthy:

   kubectl get events --sort-by=.lastTimestamp
   kubectl events --for node/<name>

   Use the namespace and time filters supported by your kubectl version when the cluster is busy. Plural's kubectl get events guide covers practical event filtering. Events provide timing; the node Conditions section provides the current state.

4. Verify the Kubelet heartbeat and local service

   Because the Kubelet reports node status to the API server, a stopped, unhealthy, or disconnected Kubelet can leave the node NotReady. Connect to the host and inspect its service logs:

   sudo systemctl status kubelet
   sudo journalctl -u kubelet --since "30 minutes ago" --no-pager

   Look for certificate errors, API server timeouts, container runtime failures, and repeated registration or heartbeat errors. Also verify that the host can reach the API server and that required CNI paths are functioning. If the node cannot resolve or connect to the control-plane endpoint, a healthy-looking Kubelet process cannot report Ready.

Record the condition that best explains the transition before taking corrective action. That evidence makes the next step safer, particularly in regulated environments where an unexplained reboot or eviction can complicate incident review.

How to Fix and Restore a NotReady Kubernetes Node

Remediation should follow the condition that caused the node to fail, not a generic reboot-first workflow. Start by preserving evidence, then restore the node only after you understand whether the failure is in the Kubelet, the host, the network path, or the container runtime. Kubernetes does not schedule new pods to a NotReady node, and the controller manager can evict existing pods after the node remains NotReady for about five minutes. See the Kubernetes cluster debugging guidance before making changes in a production environment.

Restart the Kubelet and clear host pressure

If Kubelet logs show a crashed process, configuration error, or missed heartbeat, restart it on the node:

sudo systemctl restart kubelet
sudo systemctl status kubelet
journalctl -u kubelet --since "15 minutes ago"

For a deeper overview of the component responsible for reporting node health, review this guide to Kubelet troubleshooting. Then check the node conditions again. DiskPressure requires freeing space from container images, logs, and unused volumes. MemoryPressure may require terminating or rescheduling memory-heavy workloads, correcting requests and limits, or adding capacity. PIDPressure usually points to too many processes, so inspect process counts and adjust PID limits only after confirming the workload is safe to change. Do not delete files from active volume paths as a shortcut.

Repair networking and the container runtime

A broken CNI plugin can prevent the node from communicating with the API server or prevent pod networking from initializing. Inspect the CNI daemonset and its logs, verify node-to-control-plane connectivity, and confirm that required network pods are running. If networking is healthy, review containerd or the configured runtime for socket errors, image-pull failures, and stalled tasks. Repair the underlying service, then allow Kubelet to reconcile pod state instead of repeatedly deleting pods without addressing the runtime failure.

Cordon, drain, and replace failing hardware

Before planned maintenance, isolate the node so the scheduler stops placing additional work on it:

kubectl cordon <node-name>
kubectl drain <node-name> --ignore-daemonsets --delete-emptydir-data
# perform maintenance, then restore service
kubectl uncordon <node-name>

Use drain carefully around disruption budgets, local storage, and stateful workloads. If the host is rebooting unexpectedly, reporting hardware errors, or cannot hold a stable Kubelet and runtime, replace it rather than extending recovery attempts. After repair or replacement, verify that the node returns to Ready=True and that workloads have recovered:

kubectl get nodes
kubectl get pods -A -o wide

For teams managing many clusters, this is a day-2 operations problem: every manual diagnosis competes with upgrade, compliance, and incident-response work. Plural's unified control plane and agent-based pull architecture help platform teams observe fleet health and coordinate repeatable remediation without central credential storage. That matters especially for regulated environments where reliability and data sovereignty are operational requirements, not optional improvements.

See how Plural simplifies Day-2 operations for enterprise Kubernetes fleets.

How Platform Teams Prevent Node NotReady Disruptions

Reliable Kubernetes operations depend less on reacting quickly to a failed node and more on removing the conditions that make failure difficult to detect, diagnose, and remediate. A single NotReady node may be an isolated infrastructure fault. Across a fleet, repeated events often indicate inconsistent maintenance, weak observability, or a lack of standardized day-2 procedures.

Platform teams can make that difference explicit by comparing reactive firefighting with a centralized operating model:

This model matters most in regulated environments. For a financial institution, healthcare provider, or government organization, node reliability is not only a capacity concern. Unplanned disruption can affect service objectives, audit evidence, change controls, and data-sovereignty requirements.

Plural provides a unified control plane for Kubernetes fleet management, bringing cluster operations, infrastructure workflows, and automation into a single pane of glass. Lightweight agents perform local execution while the self-hosted architecture keeps operational control close to the environment. That design lets platform teams standardize responses without requiring every engineer to hold broad credentials in a central system.

The goal is not to automate every decision blindly. It is to make the known path dependable: detect a failing node, collect the right evidence, apply an approved remediation, and preserve the context for review. Teams that want to build this operating model across public cloud, private cloud, or air-gapped clusters can explore Plural's Kubernetes fleet management platform.

Talk to Plural about a self-hosted control plane for regulated infrastructure.

Frequently Asked Questions

What does Node NotReady mean in Kubernetes?

It means Kubernetes no longer considers the node healthy and will not schedule new pods there. Existing pods may continue running while you investigate the node's Ready condition, kubelet health, resource pressure, and connectivity. See the Kubernetes Node Status reference.

How do I troubleshoot a Kubernetes Node NotReady status?

Start with kubectl get nodes, then run kubectl describe node <node-name> to inspect conditions and recent events. Check the kubelet service logs with journalctl -u kubelet, and verify the node's network and container runtime. These checks distinguish a missed heartbeat from disk, memory, PID, or CNI problems. Kubernetes recommends using node descriptions and status output for detailed diagnosis: cluster debugging documentation.

Why is my Kubernetes node showing NotReady after a restart?

A restart can leave the kubelet stopped or misconfigured, the container runtime unavailable, or the node unable to reach the API server. Confirm that kubelet and the runtime are running, inspect their logs, and check whether the CNI components recovered. If the node has disk, memory, or PID pressure, resolve that condition before expecting it to return to Ready.

Can a NotReady node cause pod eviction?

Yes. Existing pods are not necessarily stopped immediately, but the control plane can evict them when the node remains NotReady. Kubernetes documentation describes eviction after five minutes of NotReady status by default, so verify workload redundancy and capacity before draining, rebooting, or replacing the node: Kubernetes cluster debugging documentation.

Ready to make Kubernetes day-2 operations more predictable?

Centralizing fleet visibility and operational workflows can give platform teams a clearer path from node diagnosis to reliable recovery across clusters. Get started with Plural to bring Kubernetes fleet management and day-2 operations under one control plane.