Kubernetes Pod Stuck in Pending: How to Find and Fix the Reason
A pod stuck in Pending has no node yet, so nothing is running. Run kubectl describe pod <name> and read the FailedScheduling event. It tells you exactly why every node was rejected, for example Insufficient cpu, untolerated taint, didn't match node selector or unbound PersistentVolumeClaims. Fix that one rule, or add a node.
Quick answer
kubectl describe pod report-job-8xk2 | grep -A5 Events # read FailedScheduling
kubectl describe nodes | grep -A8 "Allocated resources" # how full is each node?
kubectl get pvc # all claims Bound?
A live example first
On a small 2-node EKS lab (t3.medium nodes, 2 vCPU each) we deploy an app that asks for 3 CPUs:
resources:
requests:
cpu: "3"
memory: "512Mi"
kubectl apply -f report.yaml
kubectl get pods
NAME READY STATUS RESTARTS AGE
report-5d9c7b8f4-hk2lp 0/1 Pending 0 3m
kubectl describe pod report-5d9c7b8f4-hk2lp
Events:
Warning FailedScheduling default-scheduler 0/2 nodes are available:
2 Insufficient cpu. preemption: 0/2 nodes are available:
2 No preemption victims found for incoming pod.
A t3.medium has 2 vCPU, and part of that is reserved for the system. No node can ever give 3 CPUs. We change the request to 500m and apply again, and the pod starts in seconds. Friends, the scheduler always tells you the reason. You only have to read it.
Why does a pod stay Pending?
The scheduler explains itself in FailedScheduling: CPU, memory, taints, selectors or volumes.
The scheduler looks at every node and filters out the ones that cannot run the pod. A node is filtered out when:
- the free requests (not real usage) are smaller than the pod's requests,
- the node has a taint the pod does not tolerate,
- the node labels do not match the pod's nodeSelector or affinity,
- a required volume is not ready or is in another Availability Zone.
If every node is filtered out, the pod waits in Pending and the scheduler tries again later.
How do I fix a Pending pod step by step?
- Run
kubectl describe pod <name>. - Read the
FailedSchedulingmessage word by word. - Find the reason in the table below.
- Fix the pod spec or the cluster.
- Run
kubectl get pods -wuntil the pod leaves Pending.
Reason 1: Insufficient cpu or memory
Check how much is already promised on each node:
kubectl describe node ip-10-0-1-25.ap-south-1.compute.internal
Allocatable:
cpu: 1930m
memory: 3388Mi
Allocated resources:
Resource Requests Limits
cpu 1750m (90%) 2 (103%)
memory 2100Mi (61%) 3Gi (92%)
Only 180m of CPU is free. The fixes are:
- Lower the requests to what the app really needs (measure with
kubectl top pod). - Remove old Deployments you no longer use.
- Add nodes, or turn on Cluster Autoscaler or Karpenter on EKS.
Reason 2: Taints and tolerations
0/3 nodes are available: 3 node(s) had untolerated taint {dedicated: gpu}.
The nodes are reserved. Either run on other nodes, or add a toleration if this pod really belongs there:
tolerations:
- key: "dedicated"
operator: "Equal"
value: "gpu"
effect: "NoSchedule"
Reason 3: nodeSelector or affinity does not match
0/3 nodes are available: 3 node(s) didn't match Pod's node affinity/selector.
Compare the pod spec with the real labels:
kubectl get nodes --show-labels | tr ',' '\n' | grep -i zone
A common mistake is disktype: SSD in the pod while the node label is disktype=ssd. Labels are case-sensitive.
Reason 4: The PersistentVolumeClaim is not Bound
0/3 nodes are available: pod has unbound immediate PersistentVolumeClaims.
kubectl get pvc
kubectl describe pvc data-mysql-0
On EKS, check three things:
- The EBS CSI driver add-on is installed.
- A StorageClass exists (
kubectl get storageclass) and is the default, or is named in the PVC. - The EBS volume and the node are in the same Availability Zone. An EBS volume in ap-south-1a cannot attach to a node in ap-south-1b.
Reason 5: ResourceQuota or LimitRange
If a namespace has a quota, the pod may be refused before scheduling. Look at the ReplicaSet events:
kubectl describe rs -l app=report
kubectl describe quota -n reports
Same idea on every cloud
| Cloud | How to add capacity |
|---|---|
| AWS EKS | Increase node group desired size, or Karpenter / Cluster Autoscaler |
| Google GKE | Turn on node auto-provisioning or the cluster autoscaler |
| Azure AKS | az aks nodepool scale or the cluster autoscaler |
Real example: planning capacity
When Pokémon GO launched in 2016, Google Cloud's public blog post explained that traffic reached about 50 times the original estimate. The game ran on Google Kubernetes Engine, and the team had to add capacity and upgrade the cluster live. On a smaller scale, every Pending pod with Insufficient cpu is the same lesson: set honest requests, and have a plan for adding nodes.
Mistakes students make (and how to avoid them)
- Copying requests from a blog. A request of
2CPUs "to be safe" on a 2-vCPU node means the pod can never fit. Start from measured use. - Thinking idle CPU is free CPU. The scheduler adds up requests. Ten idle pods that each request 200m still block 2 full CPUs.
- Forgetting the DaemonSets. Logging, monitoring and the CNI run one pod per node and use part of every node before your app arrives.
- Using one AZ for storage and another for nodes. With EBS, use a StorageClass with
volumeBindingMode: WaitForFirstConsumer, so the volume is created in the zone where the pod is scheduled. - Running the autoscaler with max size equal to min size. Cluster Autoscaler cannot add a node if the node group maximum is already reached.
Practice lab (15 minutes)
- Deploy a pod that requests 3 CPUs on a small node and read the event.
- Taint a node with
kubectl taint nodes NODE dedicated=gpu:NoScheduleand watch new pods avoid it. - Remove the taint with the same command ending in
-.
Summary table
| FailedScheduling says | Cause | Fix |
|---|---|---|
Insufficient cpu / memory |
Requests bigger than free capacity | Lower requests or add nodes |
untolerated taint |
Node is reserved | Add toleration or use other nodes |
didn't match ... selector |
Labels do not match | Fix the label or the selector |
unbound ... PersistentVolumeClaims |
Volume not ready | StorageClass, CSI driver, same AZ |
| No events | Quota or scheduler name | describe rs, describe quota |
Quick quiz: check what you learned
What does the scheduler use to decide if a pod fits?
- The pod CPU and memory requests
- The live CPU usage on the node
- The size of the container image
- The number of Services defined
Show answer
A. Scheduling is based on requests, not on what the node is using right now.
"untolerated taint" in FailedScheduling means what?
- The pod lacks a toleration
- The image tag is missing here
- The PVC is already bound
- The probe path is incorrect
Show answer
A. A tainted node accepts only pods that tolerate that taint.
A pod waits because its PVC is not Bound. What do you check?
- The StorageClass and CSI driver
- The liveness probe timeout
- The Docker Hub rate limit
- The container exit code
Show answer
A. The volume must be provisioned and Bound before the pod can start.
Which command shows how much of a node is already requested?
- kubectl describe node NAME
- kubectl logs NAME --previous
- kubectl get secrets -A
- kubectl rollout undo NAME
Show answer
A. describe node shows Allocatable and the Allocated resources table.
Learn it properly
This guide is the short path. The free DevOps course covers Kubernetes step by step, with labs:
Frequently asked questions
What does Pending mean in Kubernetes?
The pod is accepted but the scheduler has not found a node for it yet, or the node has not started its containers. No container is running.
What is "0/3 nodes are available: 3 Insufficient cpu"?
None of the 3 nodes has enough free CPU requests for this pod. Lower the request or add nodes.
Why is my pod Pending when the node has free CPU?
Scheduling uses requests, not real usage. If other pods requested the CPU, it is counted as used even when idle.
What does "untolerated taint" mean?
The node has a taint, such as a control-plane or GPU taint, and your pod has no matching toleration.
Can a PVC keep a pod Pending?
Yes. If the PersistentVolumeClaim is not Bound, the pod waits. Check the StorageClass and the EBS CSI driver on EKS.
Pending with no events at all?
Check that the scheduler is running and that schedulerName in the pod spec is correct. On managed clusters, check the namespace ResourceQuota.