56.9 Registries: Docker Hub limits, GHCR, Harbor and pull-through mirrors
Friends, every image we build has to live somewhere, and every server has to pull it from somewhere. Docker Hub is the default, but it has pull limits, and a company usually wants its own private registry close to its servers. Today, step by step: how image names point to registries, Docker Hub limits, GitHub Container Registry, Amazon ECR, running your own registry, Harbor for companies, a pull-through mirror that caches Docker Hub, and copying images between registries.
मित्रांनो, आपण बनवलेली प्रत्येक image कुठेतरी ठेवावी लागते, आणि प्रत्येक server ला ती कुठूनतरी pull करावी लागते. Docker Hub हा default आहे, पण त्याला pull limits आहेत, आणि company ला सहसा आपल्या servers जवळ स्वतःचा private registry हवा असतो. आज step by step: image names registries कडे कसे निर्देश करतात, Docker Hub limits, GitHub Container Registry, Amazon ECR, स्वतःचा registry चालवणं, companies साठी Harbor, Docker Hub cache करणारा pull-through mirror, आणि registries मध्ये images copy करणं.
दोस्तों, हमारी बनाई हर image कहीं रखनी पड़ती है, और हर server को उसे कहीं से pull करना पड़ता है. Docker Hub default है, पर उसकी pull limits हैं, और company को आमतौर पर अपने servers के पास अपना private registry चाहिए. आज step by step: image names registries की ओर कैसे इशारा करते हैं, Docker Hub limits, GitHub Container Registry, Amazon ECR, अपना registry चलाना, companies के लिए Harbor, Docker Hub को cache करने वाला pull-through mirror, और registries के बीच images copy करना.
1. Start from a story
At Ola, the build farm had forty Jenkins agents. Every morning, as builds started, they all pulled node and python base images from Docker Hub from the same office IP address.
By ten o'clock builds started failing with toomanyrequests: You have reached your pull rate limit. Developers waited, retried and lost hours.
A platform engineer named Sameer set up two things. A pull-through mirror inside the office cached every Docker Hub image after the first pull, and Harbor became the private registry for company images, with scanning and access rules. The rate limit errors stopped the same day, and builds got faster because images came over the local network.
Today we build Sameer's setup step by step.
2. Names and Docker Hub
Step 1: How an image name finds its registry
Why: The first part of the name.
What: If the first part of an image name has a dot or a colon, like ghcr.io or registry.example.com:5000, it is the registry host. If not, Docker uses Docker Hub, and a name without a slash gets library added in front.
Example 1: Short name
Type this:
docker pull -q nginx:1.27-alpine
You will see:
docker.io/library/nginx:1.27-alpine
What it means: Docker Hub, official library.
Example 2: User repo
Type this:
docker pull -q bitnami/redis:7.4
You will see:
docker.io/bitnami/redis:7.4
What it means: Docker Hub, the bitnami namespace.
Example 3: Other registry
Type this:
docker pull -q ghcr.io/example-org/tools:1.0 2>&1 | tail -1
You will see:
ghcr.io/example-org/tools:1.0
What it means: The host part says GitHub Container Registry. Example name.
Step 2: Docker Hub rate limits
Why: Why builds fail in the morning.
What: Anonymous pulls are limited per IP address over a time window, and logged-in free accounts get a higher limit. Many agents behind one office IP hit the anonymous limit quickly. You can read your remaining quota from the response headers.
Example 4: Get a token
Type this:
TOKEN=$(curl -s 'https://auth.docker.io/token?service=registry.docker.io&scope=repository:ratelimitpreview/test:pull' | jq -r .token) && echo ${#TOKEN}
You will see:
1340
What it means: A temporary anonymous token, only its length is printed.
Example 5: Read the limit
Type this:
curl -s --head -H "Authorization: Bearer $TOKEN" https://registry-1.docker.io/v2/ratelimitpreview/test/manifests/latest | grep -i ratelimit
You will see:
ratelimit-limit: 100;w=21600
ratelimit-remaining: 12;w=21600
What it means: Only 12 pulls left in this window.
Example 6: The error
Type this:
docker pull python:3.12-slim
You will see:
toomanyrequests: You have reached your pull rate limit.
What it means: This is what failing builds show.
Step 3: Log in to Docker Hub with a token
Why: Higher limit, no password.
What: Create a personal access token in Docker Hub settings with read-only scope and log in with --password-stdin. The CI system stores the token as a secret.
Example 7: Log in
Type this:
printf '%s' "$DOCKERHUB_TOKEN" | docker login -u shopci --password-stdin
You will see:
Login Succeeded
What it means: The token comes from a CI secret. Example user.
Example 8: Where it is stored
Type this:
jq -r '.auths | keys[]' ~/.docker/config.json
You will see:
https://index.docker.io/v1/
What it means: Docker remembers the login.
Example 9: Use a credential helper
Type this:
jq -r .credsStore ~/.docker/config.json
You will see:
pass
What it means: With a helper, the token is not stored as plain text in the file.
3. Hosted private registries
Step 4: GitHub Container Registry (GHCR)
Why: Images next to your code.
What: GHCR lives at ghcr.io. Image names are ghcr.io/OWNER/NAME:TAG. Log in with a personal access token that has read and write packages scope, or with the automatic token inside GitHub Actions.
Example 10: Log in
Type this:
printf '%s' "$GHCR_TOKEN" | docker login ghcr.io -u example-user --password-stdin
You will see:
Login Succeeded
What it means: Example user and token.
Example 11: Tag and push
Type this:
docker tag shop-api:1.4 ghcr.io/example-org/shop-api:1.4 && docker push ghcr.io/example-org/shop-api:1.4
You will see:
1.4: digest: sha256:9f2c... size: 1784
What it means: The package appears under the organisation.
Example 12: Pull from a server
Type this:
docker pull -q ghcr.io/example-org/shop-api:1.4
You will see:
ghcr.io/example-org/shop-api:1.4
What it means: Private packages need the same login on the server.
Step 5: Amazon ECR recap
Why: Best inside AWS.
What: ECR is the natural choice for ECS and EKS, because servers pull with their IAM role and traffic stays in AWS.
Example 13: Login
Type this:
aws ecr get-login-password --region ap-south-1 | docker login -u AWS --password-stdin 123456789012.dkr.ecr.ap-south-1.amazonaws.com
You will see:
Login Succeeded
What it means: 12-hour login from the IAM role.
Example 14: List tags
Type this:
aws ecr list-images --repository-name shop-api --region ap-south-1 --query 'imageIds[].imageTag' --output text
You will see:
a1b2c3d e4f5a6b
What it means: Images by commit.
Example 15: ECR pull-through for Docker Hub
Type this:
aws ecr create-pull-through-cache-rule --ecr-repository-prefix dockerhub --upstream-registry-url registry-1.docker.io --credential-arn arn:aws:secretsmanager:ap-south-1:123456789012:secret:ecr-pullthroughcache/dockerhub --region ap-south-1 --query ecrRepositoryPrefix --output text
You will see:
dockerhub
What it means: Pull 123456789012.dkr.ecr.ap-south-1.amazonaws.com/dockerhub/library/nginx and ECR caches it.
4. Run your own registry
Step 6: The registry image
Why: A private registry in one command.
What: The official registry:2 image is a simple registry. Good for labs and small teams. For real use put TLS and authentication in front.
Example 16: Start it
Type this:
docker run -d --name reg -p 5000:5000 -v regdata:/var/lib/registry --restart unless-stopped registry:2
You will see:
4d3e...
What it means: Data kept in a volume.
Example 17: Push to it
Type this:
docker tag nginx:1.27-alpine localhost:5000/web:1.0 && docker push localhost:5000/web:1.0
You will see:
1.0: digest: sha256:7d1b... size: 1568
What it means: localhost is allowed without TLS.
Example 18: Read the catalog
Type this:
curl -s localhost:5000/v2/_catalog
You will see:
{"repositories":["web"]}
What it means: The registry API lists repositories.
Step 7: Add basic authentication
Why: Never leave it open.
What: Create an htpasswd file with bcrypt and point the registry at it. Combine with TLS from a reverse proxy or certificates.
Example 19: Create users
Type this:
mkdir -p auth && docker run --rm --entrypoint htpasswd httpd:2 -Bbn ci example-only-pass > auth/htpasswd && chmod 0640 auth/htpasswd && cut -d: -f1 auth/htpasswd
You will see:
ci
What it means: Bcrypt hash stored, example password.
Example 20: Start with auth
Type this:
docker rm -f reg && docker run -d --name reg -p 5000:5000 -v regdata:/var/lib/registry -v "$PWD/auth":/auth:ro -e REGISTRY_AUTH=htpasswd -e REGISTRY_AUTH_HTPASSWD_REALM=reg -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd registry:2
You will see:
reg
8f1a...
What it means: Now login is required.
Example 21: Without login
Type this:
curl -s -o /dev/null -w '%{http_code}\n' localhost:5000/v2/_catalog
You will see:
401
What it means: Unauthorized, as intended.
Step 8: TLS for your registry
Why: Docker refuses plain HTTP to remote hosts.
What: Docker only talks plain HTTP to localhost. For any other host it wants HTTPS. Give the registry a real certificate, or for a private CA put the CA file in /etc/docker/certs.d/HOST:PORT/ca.crt on each client. Avoid the insecure-registries setting outside a lab.
Example 22: The error without TLS
Type this:
docker push reg.office.example:5000/web:1.0
You will see:
http: server gave HTTP response to HTTPS client
What it means: Docker expected HTTPS.
Example 23: Trust a private CA
Type this:
sudo mkdir -p /etc/docker/certs.d/reg.office.example:5000 && sudo cp office-ca.crt /etc/docker/certs.d/reg.office.example:5000/ca.crt && ls /etc/docker/certs.d/reg.office.example:5000
You will see:
ca.crt
What it means: No daemon restart needed for certs.d.
Example 24: Start the registry with TLS
Type this:
docker run -d --name regtls -p 5000:5000 -v "$PWD/certs":/certs:ro -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/reg.crt -e REGISTRY_HTTP_TLS_KEY=/certs/reg.key registry:2 && curl -s --cacert office-ca.crt https://reg.office.example:5000/v2/
You will see:
{}
What it means: HTTPS works and the API answers.
Step 9: Delete old images and free registry disk
Why: Registries fill up too.
What: Deleting a tag only removes the pointer. The layers stay until garbage collection runs. In registry:2 enable delete, delete by digest, then run garbage-collect. ECR and Harbor do this with lifecycle or retention rules.
Example 25: Find the digest
Type this:
curl -s -I -H 'Accept: application/vnd.docker.distribution.manifest.v2+json' localhost:5000/v2/web/manifests/1.0 | grep -i docker-content-digest
You will see:
Docker-Content-Digest: sha256:7d1b...
What it means: We delete by digest, not by tag. The registry needs REGISTRY_STORAGE_DELETE_ENABLED=true.
Example 26: Delete it
Type this:
curl -s -o /dev/null -w '%{http_code}\n' -X DELETE localhost:5000/v2/web/manifests/sha256:7d1b...
You will see:
202
What it means: Accepted; the tag is gone.
Example 27: Garbage collect
Type this:
docker exec reg registry garbage-collect /etc/docker/registry/config.yml | tail -1
You will see:
blob eligible for deletion: sha256:4f4f...
What it means: Layers removed from disk. Harbor and ECR use retention rules instead.
Step 10: Harbor for companies
Why: Projects, roles, scanning, replication.
What: Harbor is a CNCF registry with a web UI, projects, role-based access, Trivy scanning, image signing, quotas and replication between sites. It installs with Docker Compose or a Helm chart.
Example 28: Prepare the installer
Type this:
tar xzf harbor-online-installer-v2.11.1.tgz && cd harbor && cp harbor.yml.tmpl harbor.yml && grep -E '^hostname' harbor.yml
You will see:
hostname: reg.example.com
What it means: Set your hostname and certificate paths.
Example 29: Install with Trivy
Type this:
sudo ./install.sh --with-trivy | tail -1
You will see:
✔ ----Harbor has been installed and started successfully.----
What it means: Runs as a Compose stack.
Example 30: Push to a project
Type this:
docker login reg.example.com -u ci --password-stdin < ci-token.txt && docker tag shop-api:1.4 reg.example.com/shop/shop-api:1.4 && docker push reg.example.com/shop/shop-api:1.4 | tail -1
You will see:
1.4: digest: sha256:9f2c... size: 1784
What it means: Harbor scans it automatically.
5. Mirrors and copying
Step 11: A pull-through mirror for Docker Hub
Why: Sameer's fix.
What: Run registry:2 with a proxy remoteurl to Docker Hub, then tell every Docker daemon to use it with registry-mirrors in daemon.json. The first pull goes to Docker Hub; every later pull comes from the cache.
Example 31: Start the mirror
Type this:
docker run -d --name mirror -p 5001:5000 -v mirrordata:/var/lib/registry -e REGISTRY_PROXY_REMOTEURL=https://registry-1.docker.io registry:2
You will see:
2c7b...
What it means: A caching proxy.
Example 32: Configure the daemon
Type this:
printf '%s\n' '{ "registry-mirrors": ["http://mirror.office.example:5001"] }' | sudo tee /etc/docker/daemon.json && sudo systemctl restart docker && docker info --format '{{.RegistryConfig.Mirrors}}'
You will see:
[http://mirror.office.example:5001/]
What it means: Docker Hub pulls go through the mirror.
Example 33: Prove the cache
Type this:
docker pull -q python:3.12-slim && curl -s localhost:5001/v2/_catalog
You will see:
docker.io/library/python:3.12-slim
{"repositories":["library/python"]}
What it means: Cached for every other agent.
Step 12: Copy images between registries
Why: Without pulling to disk.
What: skopeo and crane copy images directly between registries, keeping every platform of a multi-arch image. Useful for promoting images from staging to production registries.
Example 34: skopeo copy
Type this:
skopeo copy --all docker://ghcr.io/example-org/shop-api:1.4 docker://reg.example.com/shop/shop-api:1.4
You will see:
Copying blob 4f4fb700ef54 done
Writing manifest to image destination
What it means: All platforms copied.
Example 35: crane digest
Type this:
crane digest reg.example.com/shop/shop-api:1.4
You will see:
sha256:9f2c...
What it means: Same digest as the source.
Example 36: List tags remotely
Type this:
skopeo list-tags docker://reg.example.com/shop/shop-api | jq -r '.Tags[]'
You will see:
1.3
1.4
What it means: No pull needed.
6. Common mistakes and how Sameer avoids them
Mistake one: every agent pulling anonymously from Docker Hub. He uses a mirror and a logged-in token.
Mistake two: a private registry without TLS or authentication. He never exposes registry:2 without both.
Mistake three: passwords in docker login on the command line. Always --password-stdin.
Mistake four: copying images with pull, tag and push and losing other platforms. He uses skopeo copy --all.
Check yourself
Q1. How does Docker know which registry an image is in?
From the first part of the name if it has a dot or colon; otherwise Docker Hub.
Q2. What fixes Docker Hub rate limits for many agents?
A pull-through mirror plus logged-in pulls.
Q3. What does Harbor add over registry:2?
UI, projects, roles, scanning, signing, quotas and replication.
Q4. How do you copy a multi-arch image between registries?
skopeo copy --all or crane copy.
Friends, know how names map to registries, log in with tokens, keep company images in ECR, GHCR or Harbor, and put a pull-through mirror in front of Docker Hub. Did you understand? Let us try it once more. Start a registry with authentication, push one image, then start a mirror and prove the second pull comes from the cache.
मित्रांनो, names registries शी कसे जुळतात ते जाणा, tokens ने login करा, company images ECR, GHCR किंवा Harbor मध्ये ठेवा, आणि Docker Hub समोर pull-through mirror ठेवा. authentication सह registry सुरू करा, एक image push करा, मग mirror सुरू करून दुसरा pull cache मधून येतो हे सिद्ध करा.
दोस्तों, names registries से कैसे जुड़ते हैं यह जानिए, tokens से login कीजिए, company images ECR, GHCR या Harbor में रखिए, और Docker Hub के आगे pull-through mirror रखिए. authentication के साथ registry शुरू कीजिए, एक image push कीजिए, फिर mirror शुरू करके साबित कीजिए कि दूसरा pull cache से आता है.
Ravindra Bagale's Tip
Set up the mirror before you need it. The day you hit the rate limit is the day you have no time to build one.
Mirror गरज पडण्याआधीच सेट करा. ज्या दिवशी rate limit लागतो त्या दिवशी तो बनवायला वेळ नसतो.
Mirror ज़रूरत से पहले ही सेट कीजिए. जिस दिन rate limit लगती है उस दिन उसे बनाने का समय नहीं होता.
Lab: try at home
- Read your Docker Hub rate limit headers.
- Log in to Docker Hub with a read-only token.
- Push one image to GHCR or ECR.
- Run registry:2 with htpasswd and confirm 401 without login.
- Run a pull-through mirror and configure daemon.json.
- Copy an image with skopeo copy --all.
- Install Harbor in a lab VM if you have time.
Recap
- The first part of an image name is the registry.
- Docker Hub has pull limits; log in and mirror.
- GHCR, ECR and Harbor hold private images.
- registry:2 needs TLS and auth before real use.
- skopeo and crane copy images between registries.