How to Fix ImagePullBackOff and ErrImagePull in Kubernetes
ImagePullBackOff means the node could not download your container image, so the container never started. Run kubectl describe pod <name> and read the Failed event at the bottom. "not found" means a wrong image name or tag, "unauthorized" or "no basic auth credentials" means a missing registry login, and "toomanyrequests" means a Docker Hub rate limit.
Quick answer
kubectl describe pod web-5f7c9-abcde | tail -20 # read the Failed event
docker pull nginx:1.27.0 # test the same image yourself
kubectl create secret docker-registry regcred \
--docker-server=https://index.docker.io/v1/ \
--docker-username=ravi --docker-password='<token>' # private registry login
A live example first
We make a small typing mistake, nginx:1.277 instead of nginx:1.27:
kubectl create deployment web --image=nginx:1.277
kubectl get pods
NAME READY STATUS RESTARTS AGE
web-7b6c9d8f6c-4tq9p 0/1 ImagePullBackOff 0 48s
Note RESTARTS is 0. The container never ran, so kubectl logs shows nothing useful. The answer is in the events:
kubectl describe pod web-7b6c9d8f6c-4tq9p
Events:
Normal Pulling 45s kubelet Pulling image "nginx:1.277"
Warning Failed 43s kubelet Failed to pull image "nginx:1.277": rpc error:
... docker.io/library/nginx:1.277: not found
Warning Failed 43s kubelet Error: ErrImagePull
Normal BackOff 20s kubelet Back-off pulling image "nginx:1.277"
Warning Failed 20s kubelet Error: ImagePullBackOff
"not found" is clear. Fix the tag:
kubectl set image deployment/web nginx=nginx:1.27
kubectl get pods -w
Within seconds the pod is 1/1 Running. Friends, most ImagePullBackOff cases in class are exactly this: a typo.
Why does the image pull fail?
The Events line tells you which one it is: not found, unauthorized or toomanyrequests.
The kubelet on each node pulls the image through the container runtime (containerd). The registry answers with an HTTP status: 404 for not found, 401 for login problems, 429 for too many requests. Kubernetes shows this as ErrImagePull on the first failure and ImagePullBackOff while it waits to retry (the same 10s to 5 min back-off used for crashes).
How do I fix ImagePullBackOff step by step?
- Run
kubectl describe pod <name>. - Read the last
Failedevent line. - If it says not found or manifest unknown, check the name and tag.
- If it says unauthorized, denied or no basic auth credentials, fix the login.
- If it says toomanyrequests, fix the rate limit.
- If it says i/o timeout or dial tcp, fix node internet access (NAT gateway or VPC endpoints).
- Apply the fix and run
kubectl get pods -w.
Fixes by registry
Docker Hub (public images)
Check that the tag really exists on hub.docker.com. Official images live under library/, so nginx means docker.io/library/nginx.
Since November 2020, Docker Hub limits anonymous pulls per IP address. On a cloud cluster, all nodes behind one NAT gateway share that single IP, so a busy cluster hits the limit fast:
toomanyrequests: You have reached your pull rate limit.
Fix it with an authenticated pull secret, or copy the image into your own registry once.
Docker Hub (private repo)
kubectl create secret docker-registry regcred \
--docker-server=https://index.docker.io/v1/ \
--docker-username=ravi \
--docker-password='<access-token>'
spec:
imagePullSecrets:
- name: regcred
containers:
- name: app
image: ravi/orders-api:2.4.1
Use an access token, not your account password, and keep the secret in the same namespace as the pod.
Amazon ECR on EKS
The image name must be the full ECR URI:
123456789012.dkr.ecr.ap-south-1.amazonaws.com/orders-api:2.4.1
- Open IAM and find the EKS node group role.
- Attach the AWS managed policy
AmazonEC2ContainerRegistryReadOnly. - Check that the region in the URI is the region where you pushed.
- Check that private subnets have a NAT gateway or ECR VPC endpoints (
ecr.api,ecr.dkrand the S3 gateway).
With these in place, no imagePullSecret is needed.
Google Artifact Registry (GKE) and Azure ACR (AKS)
- GKE: give the node service account the
Artifact Registry Readerrole. - AKS: run
az aks update -n mycluster -g myrg --attach-acr myregistry.
The idea is the same on every cloud: the node identity needs read access to the registry.
Image architecture: the Mac M1 trap
If you build on an Apple Silicon Mac, the image is arm64. On an amd64 node, the pull works but the container fails with exec format error. Build for the node:
docker buildx build --platform linux/amd64 -t orders-api:2.4.1 --push .
Real examples
- Docker Hub rate limits (2020): when Docker announced the anonymous pull limits, many CI pipelines and clusters started failing with
toomanyrequests. AWS answered by launching the ECR Public Gallery, which mirrors popular images such as nginx and python. Today many teams pull base images frompublic.ecr.awsfor this reason. - The
latesttag: Kubernetes documentation warns against:latestin production because different nodes may pull different images. A fixed tag like2.4.1, or a digest, makes pulls repeatable and rollbacks easy.
Mistakes students make (and how to avoid them)
- Pushing to one region and pulling from another. An ECR repository in
us-east-1is not visible from an image URI that saysap-south-1. Copy the URI straight from the ECR console. - Creating the pull secret in the wrong namespace. Secrets are per namespace. A secret in
defaultdoes nothing for a pod inshop. - Using your Docker Hub password. Create an access token with read-only scope instead, so a leaked secret cannot push or delete images.
- Forgetting the tag after a rebuild. If CI built
2.4.2but the YAML still says2.4.1-rc, the old or missing tag is pulled. Let the pipeline write the tag into the manifest. - Private subnets with no way out. EKS nodes in private subnets need either a NAT gateway or the ECR VPC endpoints. Without them, the event says
i/o timeout, not "unauthorized".
Practice lab (15 minutes)
- Deploy
nginx:1.277and read thenot foundevent. - Fix it with
kubectl set image. - Push a small image to a private ECR repository.
- Remove the ECR read policy from a test node role and watch the
unauthorizedevent. - Put the policy back and confirm the pod starts.
Summary table
| Event message | Cause | Fix |
|---|---|---|
not found / manifest unknown |
Wrong name or tag | Correct the image and tag |
unauthorized / no basic auth credentials |
No registry login | imagePullSecret or node IAM role |
toomanyrequests |
Docker Hub rate limit | Authenticate, or mirror to ECR |
i/o timeout |
Node has no route to the registry | NAT gateway or VPC endpoints |
exec format error (after pull) |
Wrong CPU architecture | buildx --platform linux/amd64 |
Quick quiz: check what you learned
Where do you read the real reason for an image pull failure?
- Events in kubectl describe pod
- The output of kubectl top nodes
- The container logs --previous
- The Service endpoints list
Show answer
A. The container never started, so there are no logs; the Failed event holds the reason.
A pull fails with "not found". What do you check first?
- The image name and its tag
- The pod memory limit
- The node disk encryption
- The liveness probe path
Show answer
A. Not found means the registry has no image with that exact name and tag.
How does an EKS node pull from private ECR without a secret?
- Its IAM role has ECR read
- It uses Docker Hub login
- ECR images are all public
- kubectl sends your password
Show answer
A. With AmazonEC2ContainerRegistryReadOnly on the node role, the kubelet gets ECR tokens itself.
Docker Hub returns "toomanyrequests". What fixes it best?
- Log in or mirror to ECR
- Increase the CPU request
- Delete the namespace
- Change the restart policy
Show answer
A. The limit is per IP for anonymous pulls; authenticated pulls or your own registry avoid it.
Learn it properly
This guide is the short path. The free DevOps course covers Kubernetes step by step, with labs:
Frequently asked questions
What is the difference between ErrImagePull and ImagePullBackOff?
ErrImagePull is the first failed pull. ImagePullBackOff means Kubernetes failed again and is now waiting longer between tries. The fix is the same.
How do I see why the image pull failed?
Run kubectl describe pod POD and read the Events. The Failed line has the real message, such as "not found" or "unauthorized".
How do I pull from a private Docker Hub repo?
Create a secret with kubectl create secret docker-registry and add it under imagePullSecrets in the pod spec.
Does EKS need an imagePullSecret for ECR?
No, when the node IAM role has the AmazonEC2ContainerRegistryReadOnly policy. The kubelet logs in to ECR automatically.
What is "toomanyrequests" from Docker Hub?
Docker Hub limits anonymous pulls per IP. Nodes behind one NAT share that limit. Log in, or copy the image to ECR.
Should I use the latest tag?
No. Use a fixed version tag or digest so every node pulls the same image and pulls are predictable.