Chapter 58: Docker Projects Part 2: AWS RDS, DynamoDB, S3, Reel Upload, Microservices, Queues, Logging, Monitoring and Cloud Deploys

58.14 Project 30: Deploy the microservices shop as a Docker Swarm stack with secrets and configs

By Ravindra Bagale · Last updated: 25 October 2026

Why: the problem

Suppose our shop from Project 21 is growing. It runs with docker compose on one server. If that server goes down at 2 a.m., every order fails. We want three servers, so one can die and the shop keeps working. But Kubernetes feels heavy for a team of three. Docker already has a built-in cluster mode called Swarm. We join three machines with one command each. Then we deploy our compose file as a stack. Swarm keeps the number of copies we ask for, moves containers when a node dies, and updates one copy at a time. Swarm also has two things plain compose lacks. Secrets are encrypted and given to containers as files in memory. Configs hold files like nginx.conf and are spread to every node. In this project we build the cluster, put the DB password in a secret and the gateway config in a config, deploy the shop, kill a node to see self-healing, do a rolling update and a rollback, and rotate the secret.

What: every term in plain English

• Swarm — Docker's built-in cluster mode. Turned on with docker swarm init.
• Manager node — Keeps the cluster state with Raft and schedules work. Use 1, 3 or 5 managers.
• Worker node — Only runs containers given by managers.
• Service — A wanted state: image, replicas, ports. Swarm keeps it true.
• Task — One container of a service on one node.
• Stack — A group of services deployed from one compose file with docker stack deploy.
• Overlay network — A network that spans all nodes, so services talk by name across machines.
• Routing mesh — A published port works on every node and is forwarded to a healthy task.
• Secret — Encrypted data stored in the managers, mounted at /run/secrets/NAME inside allowed tasks only.
• Config — A plain file (not secret) stored in the cluster, mounted into tasks.

How: step by step

1. Prepare three nodes. Docker installed, ports 2377, 7946 and 4789 open between them.
2. Init and join. docker swarm init on node1, docker swarm join on node2 and node3.
3. Create secrets and configs. DB password as a secret, gateway nginx.conf as a config.
4. Write the stack file. Compose file with deploy: replicas, update_config, placement.
5. Deploy and test. docker stack deploy, then curl any node.
6. Break things on purpose. Drain or stop a node; watch tasks move.
7. Update, roll back, rotate, clean up. New image tag, rollback, new secret version, then leave the swarm.

Architecture

Folder layout

shop-swarm/
  cluster/init.sh  cluster/security-group.txt
  stack/shop.yaml  stack/gateway.conf  stack/gateway-v2.conf  stack/deploy.sh
  ops/drill.sh  ops/rotate-secret.sh  ops/cleanup.sh

Example 1: Create a three-node Swarm

Goal. Create a three-node Swarm.

cluster/init.sh

# on node1 (private IP 10.0.1.11)
docker swarm init --advertise-addr 10.0.1.11
docker swarm join-token manager -q > /tmp/manager-token
docker swarm join-token worker -q

# on node2 and node3, paste the worker token printed above
docker swarm join --token SWMTKN-1-xxxx-worker 10.0.1.11:2377

# back on node1
docker node ls
docker node update --label-add db=true node1

cluster/security-group.txt

Between the three nodes only (source = the same security group):
  TCP 2377   cluster management (managers)
  TCP/UDP 7946  node gossip
  UDP 4789   overlay network traffic (VXLAN)
From the internet:
  TCP 80     the shop
  TCP 22     SSH, your IP only

What each line does

• --advertise-addr — The IP other nodes use to reach this manager. Use the private IP.
• join tokens — One token for workers, one for managers. Treat them like passwords. Rotate with docker swarm join-token --rotate worker.
• Why 3 managers in real life — Raft needs a majority. With 3 managers, 1 can fail. Here we keep 1 manager and 2 workers to stay simple.
• node label db=true — Postgres keeps its data in a local volume, so we pin it to node1 with a placement rule.
• Never open 2377 to the internet — Anyone with a token could join. Keep these ports inside the security group.

Green output

node1$ docker swarm init --advertise-addr 10.0.1.11
Swarm initialized: current node (k2l9...) is now a manager.
node2$ docker swarm join --token SWMTKN-1-xxxx-worker 10.0.1.11:2377
This node joined a swarm as a worker.
node1$ docker node ls
ID          HOSTNAME   STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSION
k2l9... *   node1      Ready     Active         Leader           27.3.1
m8p1...     node2      Ready     Active                          27.3.1
q4r7...     node3      Ready     Active                          27.3.1

Expected result. Three nodes are in one cluster. node1 is the leader.

Example 2: Create secrets and configs, then deploy the shop as a stack

Goal. Create secrets and configs, then deploy the shop as a stack.

stack/gateway.conf

upstream users    { server users:3000; }
upstream orders   { server orders:8000; }
upstream payments { server payments:8080; }
server {
  listen 80;
  resolver 127.0.0.11 valid=10s;
  location /api/users/    { proxy_pass http://users/; }
  location /api/orders/   { proxy_pass http://orders/; }
  location /api/payments/ { proxy_pass http://payments/; }
  location /healthz       { return 200 "ok"; }
}

stack/shop.yaml

services:
  gateway:
    image: nginx:1.27-alpine
    ports: ["80:80"]
    configs:
      - source: gateway_v1
        target: /etc/nginx/conf.d/default.conf
    networks: [front, back]
    deploy:
      replicas: 2
      update_config: { parallelism: 1, delay: 10s, order: start-first }

  users:
    image: registry.example.com/shop/users:1.0.0
    environment:
      DB_HOST: db
      DB_USER: shop
      DB_PASSWORD_FILE: /run/secrets/db_password
    secrets: [db_password]
    networks: [back]
    deploy:
      replicas: 3
      update_config: { parallelism: 1, delay: 10s, failure_action: rollback, order: start-first }
      rollback_config: { parallelism: 1 }
      restart_policy: { condition: on-failure, max_attempts: 3 }
      resources:
        limits: { cpus: "0.50", memory: 256M }
    healthcheck:
      test: ["CMD", "wget", "-qO-", "http://localhost:3000/healthz"]
      interval: 10s
      retries: 3

  orders:
    image: registry.example.com/shop/orders:1.0.0
    environment:
      DB_HOST: db
      DB_PASSWORD_FILE: /run/secrets/db_password
    secrets: [db_password]
    networks: [back]
    deploy: { replicas: 2 }

  payments:
    image: registry.example.com/shop/payments:1.0.0
    secrets: [payments_api_key]
    networks: [back]
    deploy: { replicas: 2 }

  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: shop
      POSTGRES_PASSWORD_FILE: /run/secrets/db_password
    secrets: [db_password]
    volumes: [db-data:/var/lib/postgresql/data]
    networks: [back]
    deploy:
      replicas: 1
      placement:
        constraints: [node.labels.db == true]

networks:
  front: { driver: overlay }
  back:  { driver: overlay, attachable: true }

volumes:
  db-data:

configs:
  gateway_v1:
    file: ./gateway.conf

secrets:
  db_password:
    external: true
  payments_api_key:
    external: true

stack/deploy.sh

set -e
# secrets: read from your password manager or generate; never commit them
openssl rand -base64 24 | docker secret create db_password -
printf '%s' "$PAYMENTS_KEY" | docker secret create payments_api_key -
docker secret ls

docker stack deploy -c shop.yaml --with-registry-auth shop
docker stack services shop

What each line does

• _FILE env vars — The official postgres image reads POSTGRES_PASSWORD_FILE. Our apps read DB_PASSWORD_FILE and load the file. The password is never an env var you can see in docker inspect.
• external: true — The secret is created before the stack with docker secret create, so it is not in any file in git.
• docker secret create NAME - — The dash reads from stdin. The value never sits in shell history or a file.
• configs gateway_v1 — Configs cannot be changed in place. To change, create gateway_v2 and point the service to it. That makes rollback easy.
• order: start-first — A new task starts and becomes healthy before the old one stops.
• failure_action: rollback — If new tasks fail their health check, Swarm goes back to the old image by itself.
• resolver 127.0.0.11 — Docker's internal DNS. nginx re-resolves service names when tasks move.
• --with-registry-auth — Sends your registry login to the nodes, so workers can pull private images.
• placement constraint — Postgres runs only on node1, where its volume is. For real HA use RDS or a replicated database.

Green output

$ cd stack && PAYMENTS_KEY=test-key-for-lab bash deploy.sh
ID          NAME               CREATED
r1x...      db_password        2 seconds ago
t9z...      payments_api_key   1 second ago
Creating network shop_front
Creating network shop_back
Creating config shop_gateway_v1
Creating service shop_gateway
Creating service shop_users
Creating service shop_orders
Creating service shop_payments
Creating service shop_db
NAME            MODE         REPLICAS   IMAGE
shop_db         replicated   1/1        postgres:16-alpine
shop_gateway    replicated   2/2        nginx:1.27-alpine
shop_orders     replicated   2/2        registry.example.com/shop/orders:1.0.0
shop_payments   replicated   2/2        registry.example.com/shop/payments:1.0.0
shop_users      replicated   3/3        registry.example.com/shop/users:1.0.0
$ curl -s http://NODE3_PUBLIC_IP/api/users/healthz
ok
$ docker exec $(docker ps -q -f name=shop_users | head -1) ls /run/secrets
db_password

Expected result. The shop runs across three nodes. Any node's IP works because of the routing mesh.

Example 3: Self-healing, rolling update, rollback and secret rotation

Goal. Self-healing, rolling update, rollback and secret rotation.

ops/drill.sh

# 1. drain node3: its tasks move to other nodes
docker node update --availability drain node3
docker service ps shop_users --format '{{.Name}} {{.Node}} {{.CurrentState}}'
docker node update --availability active node3

# 2. rolling update to 1.1.0, one task at a time
docker service update --image registry.example.com/shop/users:1.1.0 --with-registry-auth shop_users
docker service ps shop_users --filter desired-state=running

# 3. manual rollback
docker service rollback shop_users

# 4. new gateway config without editing the old one
docker config create gateway_v2 stack/gateway-v2.conf
docker service update --config-rm gateway_v1 \
  --config-add source=gateway_v2,target=/etc/nginx/conf.d/default.conf shop_gateway

ops/rotate-secret.sh

set -e
# rotate the payments key with zero downtime
printf '%s' "$NEW_PAYMENTS_KEY" | docker secret create payments_api_key_v2 -
docker service update \
  --secret-rm payments_api_key \
  --secret-add source=payments_api_key_v2,target=payments_api_key \
  shop_payments
docker secret rm payments_api_key
docker secret ls

ops/cleanup.sh

docker stack rm shop
sleep 15
docker secret rm db_password payments_api_key_v2
docker config rm gateway_v1 gateway_v2
docker volume rm shop_db-data
# on node2 and node3: docker swarm leave
# on node1:           docker swarm leave --force

What each line does

• drain — No new tasks go to the node and its tasks move away. Use it before patching a server.
• docker service ps — Shows the history of each task: where it ran and why it stopped.
• service update --image — Swarm replaces one task at a time with a 10 second gap, following update_config.
• service rollback — Goes back to the previous service spec in one command.
• target=payments_api_key — The new secret is mounted under the old file name, so the app code does not change.
• Stop a node for real — Run sudo systemctl stop docker on node3. After about a minute the manager marks it Down and starts the tasks elsewhere.
• Leave in order — Workers leave first; the last manager needs --force.

Green output

$ docker node update --availability drain node3
$ docker service ps shop_users --format '{{.Name}} {{.Node}} {{.CurrentState}}'
shop_users.1 node1 Running 3 minutes ago
shop_users.2 node2 Running 3 minutes ago
shop_users.3 node2 Running 4 seconds ago
shop_users.3 node3 Shutdown 5 seconds ago
$ docker service update --image registry.example.com/shop/users:1.1.0 --with-registry-auth shop_users
overall progress: 3 out of 3 tasks
verify: Service converged
$ docker service rollback shop_users
rollback: manually requested rollback
verify: Service converged
$ NEW_PAYMENTS_KEY=test-key-v2 bash ops/rotate-secret.sh
ID        NAME                  CREATED
u2b...    payments_api_key_v2   20 seconds ago

Expected result. Traffic kept flowing through the drain, the update, the rollback and the secret rotation.

Every command, step by step (plain docker and Compose)

1. Check Docker on all nodes

   docker version --format '{{.Server.Version}}'
   

   What it does: Same version on every node is best.

   27.3.1
   

2. Init swarm

   docker swarm init --advertise-addr 10.0.1.11
   

   What it does: node1 becomes manager.

   Swarm initialized
   

3. Join workers

   docker swarm join --token TOKEN 10.0.1.11:2377
   

   What it does: Run on node2 and node3.

   joined a swarm as a worker
   

4. List nodes

   docker node ls
   

   What it does: Three Ready nodes.

   Leader
   

5. Label db node

   docker node update --label-add db=true node1
   

   What it does: For the placement rule.

   node1
   

6. Create secrets

   openssl rand -base64 24 | docker secret create db_password -
   

   What it does: Stored encrypted in Raft.

   r1x...
   

7. Deploy stack

   docker stack deploy -c shop.yaml --with-registry-auth shop
   

   What it does: Creates networks, config and services.

   Creating service shop_users
   

8. See services

   docker stack services shop
   

   What it does: Replicas like 3/3.

   shop_users replicated 3/3
   

9. Test any node

   curl http://NODE3_PUBLIC_IP/api/users/healthz
   

   What it does: Routing mesh.

   ok
   

10. Logs of a service

    docker service logs -f --tail 20 shop_orders
    

    What it does: Logs from all tasks on all nodes.

    shop_orders.1 | listening
    

11. Scale

    docker service scale shop_orders=4
    

    What it does: Four orders tasks.

    verify: Service converged
    

12. Drain node

    docker node update --availability drain node3
    

    What it does: Tasks move.

    node3
    

13. Update and rollback

    docker service rollback shop_users
    

    What it does: Back to 1.0.0.

    Service converged
    

14. Clean up

    bash ops/cleanup.sh
    

    What it does: Stack, secrets, configs, volume, swarm.

    Removing service shop_users
    

When it is red 1

Error response from daemon: Timeout was reached before node joined.

Cause. Port 2377 is blocked between nodes or the wrong IP was advertised.

Fix, step by step

1. Open 2377, 7946 and 4789 inside the security group.
2. Use the private IP in --advertise-addr.

When it is red 2

shop_users.1  ... Rejected  "No such image: registry.example.com/shop/users:1.0.0"

Cause. Workers could not pull the private image.

Fix, step by step

1. Deploy with --with-registry-auth after docker login.
2. Check the tag exists in the registry.

When it is red 3

shop_db.1 ... Pending "no suitable node (scheduling constraints not satisfied on 3 nodes)"

Cause. No node has the label db=true.

Fix, step by step

1. docker node update --label-add db=true node1.

When it is red 4

Error response from daemon: rpc error: code = InvalidArgument desc = only updates to Labels are allowed

Cause. You tried to change a config or secret in place.

Fix, step by step

1. Create a new version (gateway_v2) and swap it with service update.

When it is red 5

curl: (7) Failed to connect ... but docker service ls shows 2/2

Cause. UDP 4789 is blocked, so the overlay and routing mesh cannot forward traffic.

Fix, step by step

1. Open UDP 4789 and 7946 between nodes.
2. On some clouds also check that source/destination checks are not blocking VXLAN.

When it is red 6

docker secret create: This node is not a swarm manager.

Cause. Secrets and stacks are only managed from a manager.

Fix, step by step

1. Run the command on node1 or promote a node with docker node promote.

Practice tasks

docker swarm init --advertise-addr IP

openssl rand -base64 24 | docker secret create db_password -

docker stack deploy -c shop.yaml shop

docker service scale shop_orders=4

docker node update --availability drain node3

docker service update --config-add

docker service update --image ...

failure_action: rollback

--secret-add source=payments_api_key_v2,target=payments_api_key

bash ops/cleanup.sh

Recap

• Swarm turns a few Docker hosts into one cluster with two commands.
• A stack is a compose file deployed to the swarm with deploy settings.
• Secrets are encrypted and mounted as in-memory files; use _FILE env vars.
• Configs are versioned by name; create a new one to change it.
• Drain, rolling updates and rollback keep the shop up during maintenance.
• Open 2377, 7946 and 4789 only between nodes.

FAQ

Q. Swarm or Kubernetes?

Swarm is simpler and uses compose files. Kubernetes has a much bigger ecosystem. Learn Swarm for small teams, Kubernetes for most jobs.

Q. Is Swarm still maintained?

Yes. Swarm mode is part of Docker Engine (Mirantis also supports it commercially).

Q. Are secrets safe in docker inspect?

docker inspect shows only the secret name, never the value.

Q. Can I use build: in a stack?

No. Stacks need images from a registry; build and push first.

Q. Can compose files work for both?

Yes. docker compose ignores most deploy keys; stack deploy uses them.

Q. What about volumes across nodes?

Local volumes stay on one node. Use placement, NFS or a managed database.

Q. How many managers for production?

Three or five, spread across zones.

Interview questions

1. What is Docker Swarm?

   Model answer: Docker's native orchestration mode that clusters engines and runs services with desired state.

2. Difference between a service and a task?

   Model answer: A service is the wanted state; a task is one container scheduled on a node.

3. How are Swarm secrets stored and delivered?

   Model answer: Encrypted in the Raft log on managers and mounted in memory at /run/secrets only for granted services.

4. Explain the routing mesh.

   Model answer: Published ports listen on all nodes and are load balanced to healthy tasks over the ingress network.

5. How do you do zero-downtime updates in Swarm?

   Model answer: update_config with parallelism, start-first order, health checks and failure_action rollback.

Samajla ka?

Ravi Sir's Cards for this lesson

Revision cardInterview card

मराठी cards