53.11 Separate dev and prod Compose files, deployed to an Ubuntu VM
Friends, on the laptop we want live reload, debug logs and easy ports. On the server we want fixed image tags, restart policies, limits and no source code mounts. Today we split one project into a base file, a dev file and a prod file, then deploy it to a fresh Ubuntu 24.04 VM step by step, from SSH to a working website.
मित्रांनो, laptop वर आपल्याला live reload, debug logs आणि सोपे ports हवे. Server वर fixed image tags, restart policy, limits आणि source code mount नको. आज एक project base file, dev file आणि prod file मध्ये विभागू, मग fresh Ubuntu 24.04 VM वर SSH पासून चालू website पर्यंत step by step deploy करू.
दोस्तों, laptop पर हमें live reload, debug logs और आसान ports चाहिए. Server पर fixed image tags, restart policy, limits और कोई source code mount नहीं. आज एक project को base file, dev file और prod file में बाँटेंगे, फिर fresh Ubuntu 24.04 VM पर SSH से चालू website तक step by step deploy करेंगे.
1. Start from a story
At Nykaa, a small team ran an internal offers dashboard. The same compose.yaml was used on laptops and on the EC2 server. It mounted ./src, ran nodemon and published Postgres on port 5432.
One day a security scan found Postgres open to the internet with a weak password. Another day the server ran out of disk because nodemon logs were never rotated.
Ankit, the DevOps engineer, split the file. compose.yaml kept what is the same everywhere. compose.override.yaml kept laptop extras. compose.prod.yaml added restart policies, limits, log rotation and fixed image tags, and removed every published port except 80 and 443.
Then he wrote a short deploy routine for the Ubuntu VM: install Docker, copy two files, pull, up --wait. Today we follow Ankit's steps exactly.
2. Split the files
Step 1: The base file
Why: What never changes.
What: Service names, images, networks, volumes and healthchecks live in compose.yaml. No ports, no mounts of source code.
Example 1: Write base
Type this:
printf 'services:\n api:\n image: ${REGISTRY:-local}/offers-api:${TAG:-dev}\n environment:\n DATABASE_URL: postgres://offers:${DB_PASSWORD}@db:5432/offers\n depends_on:\n db: {condition: service_healthy}\n db:\n image: postgres:16-alpine\n environment: {POSTGRES_USER: offers, POSTGRES_PASSWORD: "${DB_PASSWORD}", POSTGRES_DB: offers}\n volumes: ["pgdata:/var/lib/postgresql/data"]\n healthcheck: {test: ["CMD-SHELL","pg_isready -U offers"], interval: 5s, retries: 10}\nvolumes: {pgdata: {}}\n' > compose.yaml && docker compose config --services
You will see:
api
db
What it means: Two services, no ports yet.
Example 2: Variables
Type this:
grep -o '\${[A-Z_]*' compose.yaml | sort -u
You will see:
${DB_PASSWORD
${REGISTRY
${TAG
What it means: Three values change per place.
Example 3: Validate
Type this:
DB_PASSWORD=example-only docker compose -f compose.yaml config -q && echo valid
You will see:
valid
What it means: Base alone is valid.
Step 2: The dev override
Why: Laptop comforts.
What: compose.override.yaml builds locally, mounts src, runs the dev target and publishes ports for debugging.
Example 4: Write dev
Type this:
printf 'services:\n api:\n build: {context: ., target: dev}\n ports: ["3000:3000"]\n volumes: ["./src:/app/src"]\n environment: {LOG_LEVEL: debug}\n db:\n ports: ["5432:5432"]\n' > compose.override.yaml && echo written
You will see:
written
What it means: Ports for your tools, only on the laptop.
Example 5: Dev env
Type this:
printf 'DB_PASSWORD=example-only\n' > .env && docker compose config | grep -E 'LOG_LEVEL|target'
You will see:
LOG_LEVEL: debug
target: dev
What it means: Override merged automatically.
Example 6: Up
Type this:
docker compose up -d --wait
You will see:
✔ Container offers-db-1 Healthy
✔ Container offers-api-1 Healthy
What it means: Laptop stack running.
Step 3: The prod file
Why: Server rules.
What: compose.prod.yaml adds restart, limits, log rotation, nginx with ports 80 and 443, and nothing else.
Example 7: Write prod
Type this:
printf 'services:\n api:\n restart: unless-stopped\n environment: {LOG_LEVEL: info, NODE_ENV: production}\n deploy:\n resources: {limits: {cpus: "1.0", memory: 512M}}\n logging: {driver: json-file, options: {max-size: 10m, max-file: "3"}}\n db:\n restart: unless-stopped\n logging: {driver: json-file, options: {max-size: 10m, max-file: "3"}}\n web:\n image: nginx:1.27-alpine\n restart: unless-stopped\n ports: ["80:80", "443:443"]\n volumes: ["./nginx.conf:/etc/nginx/conf.d/default.conf:ro"]\n depends_on: [api]\n' > compose.prod.yaml && echo written
You will see:
written
What it means: Only differences.
Example 8: Merged
Type this:
DB_PASSWORD=x TAG=1.4.0 REGISTRY=registry.example.com docker compose -f compose.yaml -f compose.prod.yaml config | grep -E 'image:|published'
You will see:
image: registry.example.com/offers-api:1.4.0
image: postgres:16-alpine
image: nginx:1.27-alpine
published: "80"
published: "443"
What it means: Fixed tag; only web ports.
Example 9: No dev leaks
Type this:
docker compose -f compose.yaml -f compose.prod.yaml config | grep -cE 'target: dev|/app/src|5432:'
You will see:
0
What it means: Nothing from the laptop file.
Step 4: Build and push a fixed tag
Why: The server never builds.
What: Build the prod target on your laptop or CI, tag it with a version, and push to a registry.
Example 10: Build
Type this:
docker build --target prod -t registry.example.com/offers-api:1.4.0 . | tail -1
You will see:
naming to registry.example.com/offers-api:1.4.0 done
What it means: Version tag, never latest.
Example 11: Push
Type this:
docker push registry.example.com/offers-api:1.4.0 | tail -1
You will see:
1.4.0: digest: sha256:4be1c0... size: 1784
What it means: Now any server can pull it.
Example 12: Record
Type this:
printf 'TAG=1.4.0\n' > deploy.env && cat deploy.env
You will see:
TAG=1.4.0
What it means: The release is one line.
3. Prepare the Ubuntu 24.04 VM
Step 5: SSH in
Why: Log in with a key.
What: Use the key the cloud gave you. Never use password login on servers.
Example 13: Connect
Type this:
ssh -i ~/.ssh/offers.pem ubuntu@203.0.113.10 'lsb_release -ds'
You will see:
Ubuntu 24.04.1 LTS
What it means: 203.0.113.10 is an example address.
Example 14: Key permission
Type this:
chmod 600 ~/.ssh/offers.pem && ls -l ~/.ssh/offers.pem | cut -c1-10
You will see:
-rw-------
What it means: SSH refuses keys that others can read.
Example 15: Update
Type this:
ssh ubuntu@203.0.113.10 'sudo apt-get update -qq && sudo apt-get upgrade -y -qq && echo updated'
You will see:
updated
What it means: Patch first.
Step 6: Install Docker Engine
Why: Official packages.
What: Add Docker's apt repository and install the engine and the Compose plugin.
Example 16: Repository
Type this:
sudo install -m 0755 -d /etc/apt/keyrings && curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg && echo "deb [signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu noble stable" | sudo tee /etc/apt/sources.list.d/docker.list
You will see:
deb [signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu noble stable
What it means: noble is the 24.04 code name.
Example 17: Install
Type this:
sudo apt-get update -qq && sudo apt-get install -y -qq docker-ce docker-ce-cli containerd.io docker-compose-plugin && docker compose version
You will see:
Docker Compose version v2.29.7
What it means: Engine plus Compose.
Example 18: Enable
Type this:
sudo systemctl enable --now docker && systemctl is-active docker
You will see:
active
What it means: Starts on every boot.
Step 7: Deploy user and firewall
Why: Least privilege.
What: Add the user to the docker group and allow only SSH and web traffic.
Example 19: Group
Type this:
sudo usermod -aG docker ubuntu && newgrp docker && docker ps
You will see:
CONTAINER ID IMAGE COMMAND
What it means: Docker without sudo. Note: the docker group is root-equivalent; give it only to deploy users.
Example 20: Firewall
Type this:
sudo ufw allow 22/tcp && sudo ufw allow 80/tcp && sudo ufw allow 443/tcp && sudo ufw --force enable && sudo ufw status | head -4
You will see:
Status: active
To Action From
22/tcp ALLOW Anywhere
What it means: Database port stays closed.
Example 21: Cloud rule
Type this:
printf 'Also set the EC2 security group to 22 from your IP, 80 and 443 from anywhere.\n'
You will see:
Also set the EC2 security group to 22 from your IP, 80 and 443 from anywhere.
What it means: Two layers of firewall.
Step 8: Project folder and secrets
Why: Where things live.
What: Create /srv/offers, copy only the compose files and nginx.conf, and write the .env with safe permissions.
Example 22: Folder
Type this:
sudo mkdir -p /srv/offers && sudo chown ubuntu:ubuntu /srv/offers && echo ready
You will see:
ready
What it means: Owned by the deploy user.
Example 23: Copy files
Type this:
scp compose.yaml compose.prod.yaml nginx.conf deploy.env ubuntu@203.0.113.10:/srv/offers/
You will see:
compose.yaml 100%
compose.prod.yaml 100%
nginx.conf 100%
deploy.env 100%
What it means: No source code, no override file.
Example 24: Server .env
Type this:
ssh ubuntu@203.0.113.10 "cd /srv/offers && printf 'DB_PASSWORD=example-only-change-me\nREGISTRY=registry.example.com\nCOMPOSE_FILE=compose.yaml:compose.prod.yaml\n' > .env && chmod 600 .env && ls -l .env | cut -c1-10"
You will see:
-rw-------
What it means: Only the owner reads it. Use a real random value on your server.
4. Deploy and update
Step 9: First deploy
Why: Pull, then up.
What: With COMPOSE_FILE in .env, plain docker compose uses base plus prod.
Example 25: Pull
Type this:
cd /srv/offers && docker compose --env-file .env --env-file deploy.env pull -q && echo pulled
You will see:
pulled
What it means: Images downloaded.
Example 26: Up
Type this:
docker compose --env-file .env --env-file deploy.env up -d --wait
You will see:
✔ Container offers-db-1 Healthy
✔ Container offers-api-1 Healthy
✔ Container offers-web-1 Started
What it means: Everything healthy.
Example 27: Test
Type this:
curl -s -o /dev/null -w '%{http_code}\n' http://203.0.113.10/
You will see:
200
What it means: Website live.
Step 10: A release script
Why: One command per release.
What: A small script updates the tag, pulls and restarts only what changed.
Example 28: Script
Type this:
printf '#!/bin/sh\nset -e\ncd /srv/offers\nprintf "TAG=%%s\\n" "$1" > deploy.env\ndocker compose --env-file .env --env-file deploy.env pull -q\ndocker compose --env-file .env --env-file deploy.env up -d --wait\n' > release.sh && chmod 755 release.sh && echo ok
You will see:
ok
What it means: Mode 755: owner writes, everyone runs.
Example 29: Release
Type this:
./release.sh 1.4.1
You will see:
✔ Container offers-api-1 Healthy
What it means: Only api was recreated.
Example 30: Rollback
Type this:
./release.sh 1.4.0 && docker compose ps api --format '{{.Image}}'
You will see:
registry.example.com/offers-api:1.4.0
What it means: Rollback is the same script with the old tag.
Step 11: Watch the server after deploy
Why: Check before you walk away.
What: Look at status, recent logs and resource use for a few minutes after every release.
Example 31: Status
Type this:
docker compose ps --format '{{.Service}} {{.Status}}'
You will see:
api Up 2 minutes (healthy)
db Up 2 minutes (healthy)
web Up 2 minutes
What it means: All healthy.
Example 32: Errors only
Type this:
docker compose logs --since 5m api | grep -ci error
You will see:
0
What it means: No errors since the release.
Example 33: Memory
Type this:
docker compose stats --no-stream --format '{{.Name}} {{.MemPerc}}'
You will see:
offers-api-1 16.4%
offers-db-1 3.1%
offers-web-1 0.2%
What it means: Well under the 512M limit.
Step 12: Survive reboots and keep it clean
Why: Production habits.
What: restart: unless-stopped brings containers back after reboot. Prune old images so the disk does not fill.
Example 34: Reboot
Type this:
sudo reboot; sleep 60; ssh ubuntu@203.0.113.10 'docker compose -f /srv/offers/compose.yaml ps --format "{{.Service}} {{.Status}}"'
You will see:
api Up 40 seconds (healthy)
db Up 41 seconds (healthy)
web Up 39 seconds
What it means: All back.
Example 35: Prune
Type this:
docker image prune -af --filter 'until=168h' | tail -1
You will see:
Total reclaimed space: 1.2GB
What it means: Old images older than a week removed.
Example 36: Disk
Type this:
df -h / | tail -1
You will see:
/dev/root 30G 7.9G 22G 27% /
What it means: Plenty of room.
5. Common mistakes and how Ankit avoids them
Mistake one: one file for laptop and server. Ankit keeps dev and prod differences in separate small files.
Mistake two: building on the server. He builds once in CI and pulls a fixed tag.
Mistake three: publishing the database port in production. Only nginx has ports.
Mistake four: a world-readable .env. He sets chmod 600 and never commits it.
Mistake five: using latest. He uses version tags so rollback is one line.
Check yourself
Q1. Which file holds laptop-only settings?
compose.override.yaml.
Q2. How does the server avoid the override file?
COMPOSE_FILE lists only compose.yaml and compose.prod.yaml.
Q3. Why push a fixed tag?
So the server pulls exactly what was tested and can roll back.
Q4. Which ports should the VM firewall allow?
22, 80 and 443.
Friends, one base, small dev and prod files, a clean VM, and a release script. Did you understand? Let us try it once more. Create a free-tier Ubuntu VM and deploy your stack with these exact steps.
मित्रांनो, एक base, छोट्या dev आणि prod files, clean VM, आणि release script. समजलं का? चला, पुन्हा एकदा करून पाहू. Free-tier Ubuntu VM बनवा आणि याच steps ने तुमचा stack deploy करा.
दोस्तों, एक base, छोटी dev और prod files, clean VM, और release script. समझ आया? चलिए, एक बार फिर करके देखते हैं। Free-tier Ubuntu VM बनाइए और इन्हीं steps से अपना stack deploy कीजिए.
Ravindra Bagale's Tip
Keep a CHANGELOG line for every tag you deploy. When a rollback is needed at night, you will know the last good version at a glance.
Deploy केलेल्या प्रत्येक tag साठी CHANGELOG मध्ये एक line ठेवा. रात्री rollback लागला तर शेवटचा चांगला version लगेच कळतो.
Deploy किए हर tag के लिए CHANGELOG में एक line रखिए. रात को rollback चाहिए तो आखिरी अच्छा version तुरंत पता चलता है.
Lab: try at home
- Split a project into base, override and prod files.
- Compare dev and prod merged output with config.
- Build and push a version tag.
- Install Docker on an Ubuntu 24.04 VM.
- Deploy with COMPOSE_FILE and up --wait.
- Release a new tag and roll back.
Recap
- Base file holds shared truth.
- Override file holds laptop extras.
- Prod file holds restart, limits, logs and only web ports.
- The server pulls fixed tags and never builds.
- Secrets live in a chmod 600 .env.
- A release script makes deploy and rollback one command.
Revision card
Interview card