Chapter 58: Docker Projects Part 2: AWS RDS, DynamoDB, S3, Reel Upload, Microservices, Queues, Logging, Monitoring and Cloud Deploys

58.13 Project 29: Push to ECR and run a 3-tier app on ECS Fargate with RDS and S3

By Ravindra Bagale · Last updated: 25 October 2026

Why: the problem

Suppose we run a small online bookstore from Pune. In Project 28 we deployed with docker context to one EC2 server. It works, but now we have a problem. When that one server dies, the shop is down. When the sale starts, one server is not enough. And somebody must patch the server every month. AWS has a service that runs our containers without any server to manage. It is ECS with Fargate. We give it an image and say how much CPU and memory. AWS runs it, restarts it if it crashes, and spreads copies across zones. Real companies like Swiggy, Zomato and Samsung use ECS for many services. In this project we push our images to ECR, the AWS private registry. Then we run the frontend and API on Fargate behind a load balancer, keep data in RDS Postgres, and store book covers in S3. The containers get AWS access through a task role, and the database password comes from Secrets Manager. No keys and no passwords in the image.

What: every term in plain English

• ECR — Elastic Container Registry. A private Docker registry inside your AWS account.
• ECS — Elastic Container Service. AWS runs and watches your containers.
• Fargate — A way to run ECS tasks without EC2 servers. You pay per vCPU and GB per second.
• Task definition — A JSON recipe: image, CPU, memory, ports, env, secrets, logs. Like a compose service.
• Service — Keeps N tasks running, replaces dead ones and connects them to a load balancer.
• ALB — Application Load Balancer. Sends /api to the API tasks and everything else to the web tasks.
• Task role — IAM role the app code uses, for example to write to S3.
• Execution role — IAM role ECS itself uses to pull from ECR, read secrets and write logs.
• awsvpc mode — Every Fargate task gets its own private IP and its own security group.

How: step by step

1. Create ECR repos and push images. One repo for web, one for api, tag with the git commit.
2. Create the data layer. RDS Postgres in private subnets and an S3 bucket for covers.
3. Store the DB password. Secrets Manager, read by ECS at task start.
4. Create roles and logs. Execution role, task role, CloudWatch log group.
5. Register task definitions. One for api, one for web.
6. Create the cluster, ALB and services. Two tasks each, health checks, path rules.
7. Roll out, scale and clean up. New tag with rolling update, autoscaling, then delete everything.

Architecture

Folder layout

bookstore/
  web/  (React, multi-stage, nginx)   api/  (Node, Dockerfile)
  iam/  ecs-trust.json  task-role-policy.json
  ecs/  api-task.json  web-task.json
  scripts/  push-ecr.sh  data.sh  services.sh  rollout.sh  cleanup.sh

Example 1: Build both images and push them to ECR with a commit tag

Goal. Build both images and push them to ECR with a commit tag.

scripts/push-ecr.sh

set -e
REGION=ap-south-1
ACCOUNT=$(aws sts get-caller-identity --query Account --output text)
REG=$ACCOUNT.dkr.ecr.$REGION.amazonaws.com
TAG=$(git rev-parse --short HEAD)

for r in bookstore-web bookstore-api; do
  aws ecr describe-repositories --repository-names $r --region $REGION >/dev/null 2>&1 \
    || aws ecr create-repository --repository-name $r --region $REGION \
         --image-scanning-configuration scanOnPush=true \
         --image-tag-mutability IMMUTABLE >/dev/null
done

aws ecr get-login-password --region $REGION | docker login --username AWS --password-stdin $REG

docker build --platform linux/amd64 -t $REG/bookstore-api:$TAG ./api
docker build --platform linux/amd64 -t $REG/bookstore-web:$TAG ./web
docker push $REG/bookstore-api:$TAG
docker push $REG/bookstore-web:$TAG
echo $TAG > .tag
echo "pushed $TAG"

api/Dockerfile

FROM node:20-alpine AS deps
WORKDIR /app
COPY package*.json ./
RUN npm ci --omit=dev

FROM node:20-alpine
WORKDIR /app
ENV NODE_ENV=production
COPY --from=deps /app/node_modules ./node_modules
COPY src ./src
USER node
EXPOSE 3000
HEALTHCHECK --interval=15s --timeout=3s CMD wget -qO- http://localhost:3000/api/healthz || exit 1
CMD ["node", "src/server.js"]

What each line does

• scanOnPush=true — ECR scans every pushed image for known CVEs. Results show in the console and in describe-image-scan-findings.
• IMMUTABLE tags — Nobody can push a different image with the same tag. 9f3c2ab always means the same bytes.
• --platform linux/amd64 — Fargate tasks in this project run on x86. Apple laptops build ARM by default.
• get-login-password — Gives a 12-hour token. No long-lived registry password.

Green output

$ bash scripts/push-ecr.sh
Login Succeeded
 => [api 2/3] RUN npm ci --omit=dev          6.1s
 => pushing 123456789012.dkr.ecr.ap-south-1.amazonaws.com/bookstore-api:9f3c2ab
9f3c2ab: digest: sha256:4be1... size: 1785
9f3c2ab: digest: sha256:a01c... size: 1574
pushed 9f3c2ab
$ aws ecr describe-image-scan-findings --repository-name bookstore-api --image-id imageTag=9f3c2ab --query 'imageScanFindings.findingSeverityCounts'
{ "LOW": 2 }

Expected result. Both images are in ECR with the commit tag, and the scan shows no high issues.

Example 2: Create RDS, the S3 bucket, the secret, roles and log group

Goal. Create RDS, the S3 bucket, the secret, roles and log group.

scripts/data.sh

set -e
REGION=ap-south-1
VPC=$(aws ec2 describe-vpcs --filters Name=isDefault,Values=true --query 'Vpcs[0].VpcId' --output text)
SUBNETS=$(aws ec2 describe-subnets --filters Name=vpc-id,Values=$VPC --query 'Subnets[0:2].SubnetId' --output text)

# security groups: alb -> tasks -> db
ALB_SG=$(aws ec2 create-security-group --group-name bs-alb --description alb --vpc-id $VPC --query GroupId --output text)
APP_SG=$(aws ec2 create-security-group --group-name bs-app --description app --vpc-id $VPC --query GroupId --output text)
DB_SG=$(aws ec2 create-security-group --group-name bs-db --description db --vpc-id $VPC --query GroupId --output text)
aws ec2 authorize-security-group-ingress --group-id $ALB_SG --protocol tcp --port 80 --cidr 0.0.0.0/0
aws ec2 authorize-security-group-ingress --group-id $APP_SG --protocol tcp --port 3000 --source-group $ALB_SG
aws ec2 authorize-security-group-ingress --group-id $APP_SG --protocol tcp --port 80 --source-group $ALB_SG
aws ec2 authorize-security-group-ingress --group-id $DB_SG --protocol tcp --port 5432 --source-group $APP_SG

# password generated by AWS and kept in Secrets Manager
SECRET_ARN=$(aws secretsmanager create-secret --name bookstore/db \
  --secret-string "$(aws secretsmanager get-random-password --exclude-punctuation --password-length 24 --query RandomPassword --output text)" \
  --query ARN --output text)

aws rds create-db-instance --db-instance-identifier bookstore-db \
  --engine postgres --engine-version 16 --db-instance-class db.t4g.micro \
  --allocated-storage 20 --master-username bookstore \
  --master-user-password "$(aws secretsmanager get-secret-value --secret-id bookstore/db --query SecretString --output text)" \
  --vpc-security-group-ids $DB_SG --no-publicly-accessible --backup-retention-period 1
aws rds wait db-instance-available --db-instance-identifier bookstore-db

BUCKET=bookstore-covers-$RANDOM$RANDOM
aws s3api create-bucket --bucket $BUCKET --region $REGION --create-bucket-configuration LocationConstraint=$REGION
aws s3api put-public-access-block --bucket $BUCKET --public-access-block-configuration BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true

aws logs create-log-group --log-group-name /ecs/bookstore
aws logs put-retention-policy --log-group-name /ecs/bookstore --retention-in-days 7
printf "VPC=%s\nSUBNETS='%s'\nALB_SG=%s\nAPP_SG=%s\nBUCKET=%s\nSECRET_ARN=%s\n" $VPC "$SUBNETS" $ALB_SG $APP_SG $BUCKET $SECRET_ARN > .infra

iam/task-role-policy.json

{
  "Version": "2012-10-17",
  "Statement": [
    { "Effect": "Allow",
      "Action": ["s3:PutObject", "s3:GetObject"],
      "Resource": "arn:aws:s3:::bookstore-covers-*/covers/*" }
  ]
}

What each line does

• Three security groups — Internet reaches only the ALB. Only the ALB reaches the tasks. Only the tasks reach Postgres. Every layer is locked to the one before it.
• --no-publicly-accessible — RDS has no public IP. You cannot reach it from the internet at all.
• get-random-password — AWS generates the password. It never appears in a file in our repo.
• Block public access — Covers are private. The API gives short presigned URLs, like Project 19.
• Task role policy — The app may only put and get objects under covers/. Least privilege.
• Execution role — Created with the AWS managed policy AmazonECSTaskExecutionRolePolicy plus a small policy allowing secretsmanager:GetSecretValue on bookstore/db-*.

Green output

$ bash scripts/data.sh
{ "Return": true, ... }
waiting for bookstore-db ... (about 6 minutes)
{ "Location": "http://bookstore-covers-1834219920.s3.amazonaws.com/" }
$ cat .infra
VPC=vpc-0a1b2c3d
SUBNETS='subnet-011 subnet-022'
ALB_SG=sg-0aa1
APP_SG=sg-0bb2
BUCKET=bookstore-covers-1834219920
SECRET_ARN=arn:aws:secretsmanager:ap-south-1:123456789012:secret:bookstore/db-Ab12Cd
$ aws rds describe-db-instances --db-instance-identifier bookstore-db --query 'DBInstances[0].Endpoint.Address' --output text
bookstore-db.c9xyz.ap-south-1.rds.amazonaws.com

Expected result. The data layer is ready and private. Nothing is open except the ALB port 80.

Example 3: Register task definitions, create the ALB and services, then roll out a new version

Goal. Register task definitions, create the ALB and services, then roll out a new version.

ecs/api-task.json

{
  "family": "bookstore-api",
  "requiresCompatibilities": ["FARGATE"],
  "networkMode": "awsvpc",
  "cpu": "256",
  "memory": "512",
  "executionRoleArn": "arn:aws:iam::123456789012:role/bookstoreExecRole",
  "taskRoleArn": "arn:aws:iam::123456789012:role/bookstoreTaskRole",
  "runtimePlatform": { "cpuArchitecture": "X86_64", "operatingSystemFamily": "LINUX" },
  "containerDefinitions": [
    {
      "name": "api",
      "image": "123456789012.dkr.ecr.ap-south-1.amazonaws.com/bookstore-api:9f3c2ab",
      "portMappings": [{ "containerPort": 3000 }],
      "environment": [
        { "name": "DB_HOST", "value": "bookstore-db.c9xyz.ap-south-1.rds.amazonaws.com" },
        { "name": "DB_USER", "value": "bookstore" },
        { "name": "DB_NAME", "value": "postgres" },
        { "name": "COVERS_BUCKET", "value": "bookstore-covers-1834219920" },
        { "name": "AWS_REGION", "value": "ap-south-1" }
      ],
      "secrets": [
        { "name": "DB_PASSWORD", "valueFrom": "arn:aws:secretsmanager:ap-south-1:123456789012:secret:bookstore/db-Ab12Cd" }
      ],
      "healthCheck": {
        "command": ["CMD-SHELL", "wget -qO- http://localhost:3000/api/healthz || exit 1"],
        "interval": 15, "timeout": 5, "retries": 3, "startPeriod": 20
      },
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/ecs/bookstore",
          "awslogs-region": "ap-south-1",
          "awslogs-stream-prefix": "api"
        }
      }
    }
  ]
}

scripts/services.sh

set -e
. ./.infra
TAG=$(cat .tag)
aws ecs create-cluster --cluster-name bookstore >/dev/null
aws ecs register-task-definition --cli-input-json file://ecs/api-task.json >/dev/null
aws ecs register-task-definition --cli-input-json file://ecs/web-task.json >/dev/null

ALB=$(aws elbv2 create-load-balancer --name bookstore-alb --subnets $SUBNETS --security-groups $ALB_SG --query 'LoadBalancers[0].LoadBalancerArn' --output text)
TG_WEB=$(aws elbv2 create-target-group --name bs-web --protocol HTTP --port 80 --vpc-id $VPC --target-type ip --health-check-path / --query 'TargetGroups[0].TargetGroupArn' --output text)
TG_API=$(aws elbv2 create-target-group --name bs-api --protocol HTTP --port 3000 --vpc-id $VPC --target-type ip --health-check-path /api/healthz --query 'TargetGroups[0].TargetGroupArn' --output text)
L=$(aws elbv2 create-listener --load-balancer-arn $ALB --protocol HTTP --port 80 --default-actions Type=forward,TargetGroupArn=$TG_WEB --query 'Listeners[0].ListenerArn' --output text)
aws elbv2 create-rule --listener-arn $L --priority 10 --conditions Field=path-pattern,Values='/api/*' --actions Type=forward,TargetGroupArn=$TG_API >/dev/null

NET="awsvpcConfiguration={subnets=[${SUBNETS// /,}],securityGroups=[$APP_SG],assignPublicIp=ENABLED}"
aws ecs create-service --cluster bookstore --service-name api --task-definition bookstore-api \
  --desired-count 2 --launch-type FARGATE --network-configuration "$NET" \
  --load-balancers targetGroupArn=$TG_API,containerName=api,containerPort=3000 \
  --deployment-configuration "minimumHealthyPercent=100,maximumPercent=200,deploymentCircuitBreaker={enable=true,rollback=true}" >/dev/null
aws ecs create-service --cluster bookstore --service-name web --task-definition bookstore-web \
  --desired-count 2 --launch-type FARGATE --network-configuration "$NET" \
  --load-balancers targetGroupArn=$TG_WEB,containerName=web,containerPort=80 >/dev/null
aws ecs wait services-stable --cluster bookstore --services api web
aws elbv2 describe-load-balancers --load-balancer-arns $ALB --query 'LoadBalancers[0].DNSName' --output text

scripts/rollout.sh

set -e
NEW=${1:?usage: rollout.sh TAG}
aws ecs describe-task-definition --task-definition bookstore-api --query taskDefinition \
 | jq --arg t "$NEW" '.containerDefinitions[0].image |= sub(":[^:]+$"; ":"+$t)
     | {family,taskRoleArn,executionRoleArn,networkMode,containerDefinitions,requiresCompatibilities,cpu,memory,runtimePlatform}' > /tmp/td-new.json
REV=$(aws ecs register-task-definition --cli-input-json file:///tmp/td-new.json --query 'taskDefinition.revision' --output text)
aws ecs update-service --cluster bookstore --service api --task-definition bookstore-api:$REV >/dev/null
aws ecs wait services-stable --cluster bookstore --services api
echo "api now on revision $REV ($NEW)"

What each line does

• family — The name of the recipe. Each register makes a new revision: bookstore-api:1, :2, :3.
• cpu 256 / memory 512 — A quarter vCPU and half a GB. The smallest Fargate size, cheap for practice.
• secrets with valueFrom — ECS reads the password at start and passes it as an env var. It is not in the image or in the JSON as text.
• awslogs driver — Container stdout goes to CloudWatch Logs. Read it with aws logs tail.
• target-type ip — Fargate tasks have their own IP, so the target group points at IPs, not instances.
• Path rule /api/* — One ALB. React from web, API from api. The browser sees one domain, so no CORS trouble.
• min 100 / max 200 — During rollout, new tasks start first and old ones stop only when new ones are healthy. Zero downtime.
• Circuit breaker — If the new tasks keep failing, ECS stops the rollout and goes back to the last good revision by itself.

Green output

$ bash scripts/services.sh
bookstore-alb-1290345.ap-south-1.elb.amazonaws.com
$ curl -s http://bookstore-alb-1290345.ap-south-1.elb.amazonaws.com/api/healthz
{"status":"ok","db":"up","s3":"ok"}
$ curl -s -X POST -F cover=@gita.jpg http://bookstore-alb-1290345.ap-south-1.elb.amazonaws.com/api/books/7/cover
{"key":"covers/7-1729821.jpg","url":"https://bookstore-covers-1834219920.s3.ap-south-1.amazonaws.com/covers/7-1729821.jpg?X-Amz-Expires=300..."}
$ bash scripts/push-ecr.sh && bash scripts/rollout.sh $(cat .tag)
pushed 1d77e0c
api now on revision 2 (1d77e0c)

Expected result. The bookstore runs on Fargate with two copies of each tier, and the new version rolled out with zero failed requests.

Every command, step by step (plain docker and Compose)

1. Push images

   bash scripts/push-ecr.sh
   

   What it does: Creates repos, builds and pushes with commit tag.

   pushed 9f3c2ab
   

2. Build data layer

   bash scripts/data.sh
   

   What it does: RDS, S3, secret, security groups, log group.

   waiting for bookstore-db
   

3. Create roles

   aws iam create-role --role-name bookstoreExecRole --assume-role-policy-document file://iam/ecs-trust.json
   

   What it does: Trust policy for ecs-tasks.amazonaws.com.

   "RoleName": "bookstoreExecRole"
   

4. Attach exec policy

   aws iam attach-role-policy --role-name bookstoreExecRole --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
   

   What it does: ECS can pull and log.

   (no output)
   

5. Create services

   bash scripts/services.sh
   

   What it does: Cluster, ALB, target groups, two services.

   bookstore-alb-...elb.amazonaws.com
   

6. Read logs

   aws logs tail /ecs/bookstore --follow --since 5m
   

   What it does: Live container logs.

   api/api/3f1a listening on 3000
   

7. Scale out

   aws ecs update-service --cluster bookstore --service web --desired-count 4
   

   What it does: Four web copies.

   "desiredCount": 4
   

8. Roll out

   bash scripts/rollout.sh 1d77e0c
   

   What it does: New revision with zero downtime.

   api now on revision 2
   

9. Clean up

   bash scripts/cleanup.sh
   

   What it does: Deletes services, ALB, RDS, bucket, repos.

   all deleted
   

When it is red 1

CannotPullContainerError: pull image manifest has been retried ... i/o timeout

Cause. The task cannot reach ECR. It has no public IP and no NAT or VPC endpoint.

Fix, step by step

1. Set assignPublicIp=ENABLED in a public subnet, or add a NAT gateway, or create ECR and S3 VPC endpoints.

When it is red 2

ResourceInitializationError: unable to pull secrets or registry auth ... AccessDeniedException

Cause. The execution role cannot read the secret.

Fix, step by step

1. Attach the secret policy to the execution role, not the task role.
2. Check the secret ARN ends with the right suffix or use a -* wildcard.

When it is red 3

Task stopped: Essential container in task exited (exit code 1) ... ECONNREFUSED bookstore-db...:5432

Cause. The DB security group does not allow the app security group.

Fix, step by step

1. Allow 5432 from APP_SG on DB_SG.
2. Check the task uses APP_SG in its network configuration.

When it is red 4

exec /usr/local/bin/docker-entrypoint.sh: exec format error

Cause. An ARM image was pushed from an Apple laptop, but the task runs X86_64.

Fix, step by step

1. Rebuild with --platform linux/amd64, or set runtimePlatform cpuArchitecture to ARM64 (cheaper too).

Practice tasks

bash scripts/push-ecr.sh

aws ecs describe-task-definition --task-definition bookstore-api

aws ecs stop-task --cluster bookstore --task TASK_ID

aws elbv2 describe-target-health --target-group-arn TG

bash scripts/rollout.sh NEWTAG

deploymentCircuitBreaker={enable=true,rollback=true}

aws application-autoscaling put-scaling-policy

bash scripts/cleanup.sh

Recap

• ECR stores private images; use immutable commit tags and scan on push.
• Fargate runs containers with no servers to patch.
• A task definition is the recipe; a service keeps copies running behind an ALB.
• The execution role pulls images and secrets; the task role is for the app.
• Security groups chain ALB to tasks to RDS; RDS and S3 stay private.
• Rolling updates with health checks and the circuit breaker give zero downtime and safe rollback.
• Delete RDS, the ALB and the bucket after practice to avoid a bill.

FAQ

Q. Is Fargate cheaper than EC2?

For small or spiky apps, often yes, because there is no idle server. For big steady loads, EC2 capacity can be cheaper.

Q. How much will this lab cost?

A few hours cost less than 1 USD if you clean up. The ALB and RDS cost money every hour they exist.

Interview questions

1. Explain ECS task definition, task and service.

   Model answer: Recipe, running copy, and the controller that keeps the desired count running behind a load balancer.

2. What is the difference between the task role and the execution role?

   Model answer: Execution role is for ECS to pull images, read secrets and write logs; task role is for the application's AWS calls.

3. How do you pass secrets to ECS containers?

   Model answer: Secrets Manager or SSM Parameter Store referenced in the secrets section, never baked into images.

4. How do you deploy with zero downtime on ECS?

   Model answer: Rolling update with min 100 and max 200 percent, ALB health checks and the deployment circuit breaker.

Samajla ka?

Ravi Sir's Cards for this lesson

Revision cardInterview cardComparison card

मराठी cards