51.8 Load balancing with nginx across scaled containers (--scale api=3)
Friends, one api container cannot handle a dinner rush. Today we run three copies with docker compose --scale and put nginx in front to share the traffic between them, step by step.
मित्रांनो, एक api container रात्रीच्या गर्दीला पुरत नाही. आज docker compose --scale ने तीन copies चालवू आणि त्यांच्या पुढे nginx ठेवून traffic वाटून देऊ, step by step.
दोस्तों, एक api container रात की भीड़ नहीं सँभाल सकता. आज docker compose --scale से तीन copies चलाएँगे और उनके आगे nginx रखकर traffic बाँटेंगे, step by step.
1. Start from a story
It is 8 PM on a Friday at Zomato. Orders are pouring in. The single api container is at 100 percent CPU and customers see a spinning wheel.
Meera, the DevOps engineer on duty, does not buy a bigger server. She runs more copies of the same api, and puts nginx in front like a manager at a restaurant door who sends each new guest to the next free table.
Docker helps in two ways. Compose can start many copies with one flag, --scale api=3. And Docker's DNS gives all copies the same name, api, so nginx only needs to know one name.
In this lesson we build that setup on one server: a tiny api that prints its own hostname, three copies of it, and nginx in front. Then we watch the requests rotate, kill a copy, and scale up and down.
2. Build the app
Step 1: A tiny api that tells who answered
Why: We need to see which copy served each request.
What: The traefik/whoami image prints its hostname, which is the container ID.
Example 1: Pull
Type this:
docker pull traefik/whoami:v1.10
You will see:
v1.10: Pulling from traefik/whoami
Status: Downloaded newer image for traefik/whoami:v1.10
What it means: Small test image.
Example 2: Run once
Type this:
docker run -d --name w1 -p 9000:80 traefik/whoami:v1.10
You will see:
c4d2...
What it means: One copy for a test.
Example 3: Ask it
Type this:
curl -s localhost:9000 | head -1
You will see:
Hostname: c4d2e1f0a9b8
What it means: The hostname is the container ID.
Step 2: Compose file without ports on api
Why: Copies cannot share one host port.
What: If api had ports: 8000:80, the second copy would fail. Only nginx publishes a port.
Example 4: Write file
Type this:
printf 'services:\n api:\n image: traefik/whoami:v1.10\n lb:\n image: nginx:1.27\n ports: ["8080:80"]\n volumes: ["./nginx.conf:/etc/nginx/conf.d/default.conf:ro"]\n depends_on: [api]\n' > compose.yaml
You will see:
(no output)
What it means: api has no ports.
Example 5: Validate
Type this:
docker compose config --services
You will see:
api
lb
What it means: Two services.
Example 6: Port clash demo
Type this:
docker run -d -p 8000:80 traefik/whoami:v1.10 && docker run -d -p 8000:80 traefik/whoami:v1.10
You will see:
...Bind for 0.0.0.0:8000 failed: port is already allocated
What it means: Why api must not publish.
Step 3: The nginx config
Why: Tell nginx where the copies are.
What: upstream groups servers. proxy_pass sends each request to the group.
Example 7: Write config
Type this:
printf 'upstream api_pool {\n server api:80;\n}\nserver {\n listen 80;\n location / { proxy_pass http://api_pool; }\n}\n' > nginx.conf
You will see:
(no output)
What it means: One name, api.
Example 8: Check syntax
Type this:
docker run --rm -v ./nginx.conf:/etc/nginx/conf.d/default.conf:ro --add-host api:127.0.0.1 nginx:1.27 nginx -t
You will see:
nginx: configuration file /etc/nginx/nginx.conf test is successful
What it means: Config is valid.
Example 9: Show
Type this:
cat nginx.conf | head -3
You will see:
upstream api_pool {
server api:80;
}
What it means: Short and clear.
3. Scale and test
Step 4: docker compose up --scale
Why: Start many copies of one service.
What: Compose names them project-api-1, project-api-2 and so on.
Example 10: Up
Type this:
docker compose up -d --scale api=3
You will see:
Container zom-api-1 Started
Container zom-api-2 Started
Container zom-api-3 Started
Container zom-lb-1 Started
What it means: Three copies plus nginx.
Example 11: List
Type this:
docker compose ps --format '{{.Name}} {{.State}}'
You will see:
zom-api-1 running
zom-api-2 running
zom-api-3 running
zom-lb-1 running
What it means: All running.
Example 12: DNS
Type this:
docker compose exec lb getent hosts api
You will see:
172.22.0.3 api
172.22.0.4 api
172.22.0.5 api
What it means: One name, three IPs.
Step 5: Watch the rotation
Why: Prove that traffic is shared.
What: Send several requests and print only the hostname.
Example 13: Loop
Type this:
for i in 1 2 3 4 5 6; do curl -s localhost:8080 | head -1; done
You will see:
Hostname: 8a1b2c3d4e5f
Hostname: 8a1b2c3d4e5f
Hostname: 8a1b2c3d4e5f
Hostname: 8a1b2c3d4e5f
Hostname: 8a1b2c3d4e5f
Hostname: 8a1b2c3d4e5f
What it means: Always the same copy. Why?
Example 14: Reason
Type this:
docker compose exec lb nginx -T | grep 'server api'
You will see:
server api:80;
What it means: nginx resolved api once, at start, to one IP.
Example 15: Count
Type this:
for i in $(seq 30); do curl -s localhost:8080 | head -1; done | sort | uniq -c
You will see:
30 Hostname: 8a1b2c3d4e5f
What it means: No balancing yet.
Step 6: Fix: resolve the name at request time
Why: Make nginx ask Docker DNS again.
What: Use resolver 127.0.0.11 with a variable in proxy_pass, so nginx looks up api often.
Example 16: New config
Type this:
printf 'resolver 127.0.0.11 valid=5s;\nserver {\n listen 80;\n set $up http://api:80;\n location / { proxy_pass $up; }\n}\n' > nginx.conf && docker compose restart lb
You will see:
Container zom-lb-1 Started
What it means: Dynamic lookup.
Example 17: Count again
Type this:
for i in $(seq 30); do curl -s localhost:8080 | head -1; done | sort | uniq -c
You will see:
10 Hostname: 8a1b2c3d4e5f
10 Hostname: 9b2c3d4e5f6a
10 Hostname: a0c3d4e5f6b7
What it means: Shared evenly by DNS round robin.
Example 18: Header
Type this:
curl -sI localhost:8080 | grep -i server
You will see:
Server: nginx/1.27.2
What it means: nginx is the front door.
Step 7: Scale up and down live
Why: Change the number of copies without downtime.
What: Run the same command with a new number; Compose adds or removes copies.
Example 19: Up to 5
Type this:
docker compose up -d --scale api=5 --no-recreate
You will see:
Container zom-api-4 Started
Container zom-api-5 Started
What it means: Two more.
Example 20: Check
Type this:
for i in $(seq 50); do curl -s localhost:8080 | head -1; done | sort -u | wc -l
You will see:
5
What it means: Five copies answer.
Example 21: Down to 2
Type this:
docker compose up -d --scale api=2
You will see:
Container zom-api-5 Removed
Container zom-api-4 Removed
Container zom-api-3 Removed
What it means: Extra copies stop.
Step 8: When a copy dies
Why: See how DNS removes it.
What: Stopped containers leave Docker DNS within a second.
Example 22: Kill one
Type this:
docker kill zom-api-2
You will see:
zom-api-2
What it means: One copy gone.
Example 23: DNS now
Type this:
docker compose exec lb getent hosts api
You will see:
172.22.0.3 api
What it means: Only the live one.
Example 24: Traffic
Type this:
for i in $(seq 10); do curl -s -o /dev/null -w '%{http_code}\n' localhost:8080; done | sort | uniq -c
You will see:
10 200
What it means: No errors for users.
Step 9: Health checks
Why: Only healthy copies should get traffic.
What: Add a healthcheck so Compose and you can see bad copies.
Example 25: Add
Type this:
printf ' healthcheck:\n test: ["CMD","wget","-qO-","http://localhost/health"]\n interval: 10s\n' >> compose.yaml
You will see:
(no output)
What it means: Appended under api in the real file.
Example 26: Status
Type this:
docker compose ps --format '{{.Name}} {{.Health}}'
You will see:
zom-api-1 healthy
zom-api-2 healthy
What it means: Health shown.
Example 27: Note
Type this:
docker inspect zom-api-1 -f '{{.State.Health.Status}}'
You will see:
healthy
What it means: Use it in alerts.
Step 10: Pass the real client IP
Why: The api should know who called it.
What: Without headers, the api sees only nginx's IP.
Example 28: Headers
Type this:
printf ' proxy_set_header X-Real-IP $remote_addr;\n proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;\n'
You will see:
(printed lines)
What it means: Add these inside location.
Example 29: See it
Type this:
curl -s localhost:8080 | grep -i x-real-ip
You will see:
X-Real-Ip: 172.22.0.1
What it means: The api sees the caller.
Example 30: Host header
Type this:
curl -s localhost:8080 | grep -i '^Host'
You will see:
Host: api
What it means: Add proxy_set_header Host $host to keep the original.
Step 11: Logs from all copies
Why: Debug the whole pool at once.
What: docker compose logs merges output from every copy with a prefix.
Example 31: All api
Type this:
docker compose logs --tail 2 api
You will see:
zom-api-1 | Starting up on port 80
zom-api-2 | Starting up on port 80
What it means: Prefixed by copy.
Example 32: nginx access
Type this:
docker compose logs --tail 2 lb
You will see:
zom-lb-1 | 172.22.0.1 - - "GET / HTTP/1.1" 200 412
What it means: Requests at the door.
Example 33: Follow
Type this:
docker compose logs -f api
You will see:
(streams until Ctrl+C)
What it means: Live view.
Step 12: Other balancing methods
Why: Round robin is not the only choice.
What: With a fixed list of servers nginx can use least_conn or ip_hash. Compose gives each copy a stable name you can list.
Example 34: Fixed list
Type this:
printf 'upstream api_pool {\n least_conn;\n server zom-api-1:80;\n server zom-api-2:80;\n server zom-api-3:80;\n}\n' > pool.conf
You will see:
(no output)
What it means: Each copy by name.
Example 35: Sticky users
Type this:
printf 'upstream api_pool {\n ip_hash;\n server zom-api-1:80;\n server zom-api-2:80;\n}\n' > sticky.conf
You will see:
(no output)
What it means: One user always hits the same copy.
Example 36: Reload
Type this:
docker compose exec lb nginx -s reload
You will see:
(no output)
What it means: New rules without dropping requests.
Step 13: Clean up
Why: Leave the server tidy.
What: down removes containers and the project network.
Example 37: Down
Type this:
docker compose down
You will see:
Container zom-lb-1 Removed
Container zom-api-1 Removed
Network zom_default Removed
What it means: All gone.
Example 38: Leftover test
Type this:
docker rm -f w1
You will see:
w1
What it means: The one-off copy.
Example 39: Verify
Type this:
docker ps --filter name=zom -q | wc -l
You will see:
0
What it means: Nothing left.
4. Common mistakes
Publishing a port on the scaled service is the number one mistake; only the load balancer should publish. The second is a static upstream: nginx resolves names once at start, so new copies are never used until you add the resolver line. The third is forgetting health checks, so a broken copy keeps getting a share of traffic.
A fourth mistake is reloading nginx with restart during peak hours. restart drops open connections; nginx -s reload keeps them. A fifth is giving copies a fixed container_name in compose.yaml, which makes --scale fail because two containers cannot share one name.
For real production across many servers, teams move to Swarm, Kubernetes or a cloud load balancer. But the idea is exactly the same as today: many identical copies behind one front door. Learn it well here on one server, and the bigger tools will feel familiar and easy to learn.
Check yourself
Q1. Why must the api service not publish a port when scaled?
Copies cannot share one host port.
Q2. Why did all requests go to one copy at first?
nginx resolved api once at start.
Q3. Which resolver address does Docker DNS use?
127.0.0.11.
Q4. How do you go from 3 to 5 copies?
docker compose up -d --scale api=5.
Friends, many identical copies, one front door. Compose --scale makes the copies, Docker DNS gives them one name, and nginx shares the traffic. Did you understand? Let us try it once more. Now kill one copy yourself and watch the traffic stay at 200.
मित्रांनो, अनेक सारख्या copies, एकच दार. Compose --scale copies बनवतो, Docker DNS त्यांना एक नाव देतो, आणि nginx traffic वाटतो. समजलं का? चला, पुन्हा एकदा करून पाहू. आता एक copy स्वतः kill करा आणि traffic 200 वरच राहतो ते बघा.
दोस्तों, कई एक जैसी copies, एक ही दरवाज़ा. Compose --scale copies बनाता है, Docker DNS उन्हें एक नाम देता है, और nginx traffic बाँटता है. समझ आया? चलिए, एक बार फिर करके देखते हैं। अब एक copy खुद kill कीजिए और देखिए traffic 200 पर ही रहता है.
Ravindra Bagale's Tip
With nginx in Docker, always use resolver 127.0.0.11 and a variable in proxy_pass; otherwise scaling does nothing.
Docker मधल्या nginx मध्ये नेहमी resolver 127.0.0.11 आणि proxy_pass मध्ये variable वापरा, नाहीतर scale करून काही फायदा नाही.
Docker के nginx में हमेशा resolver 127.0.0.11 और proxy_pass में variable इस्तेमाल कीजिए, नहीं तो scale करने का कोई फ़ायदा नहीं.
Lab: try at home
- Write compose.yaml with api and lb.
- Scale api to 3 and count hostnames.
- Fix nginx with the resolver line and count again.
- Scale to 5, then back to 2.
- Kill one copy and check status codes.
- Add X-Real-IP and see it in the reply.
Recap
- --scale makes identical copies.
- Only the load balancer publishes a port.
- Docker DNS returns all copies for one name.
- nginx needs resolver 127.0.0.11 to see changes.
- Health checks keep bad copies visible.
- compose down cleans everything.
Revision card
Interview card