Chapter 46: Terraform 3-Tier Architecture Multi-AZ with ALBs Auto Scaling Bastion NAT and RDS

46.3 Security group chain for all tiers and the bastion

By Ravindra Bagale · Last updated: 25 October 2026

Story. Suppose we are passing the shop's security review. The rule is simple: each layer accepts traffic only from the layer in front of it, and SSH is accepted only from the bastion, and the bastion only from one admin IP. Six security groups express this, and no private group ever names 0.0.0.0/0 as a source.

Steps

1. Write security.tf.
2. Run terraform validate.
3. Search the file for 0.0.0.0/0 and confirm it is only on ext_alb ingress and on egress.

# ---- security.tf ----
resource "aws_security_group" "ext_alb" {
  name   = "${var.project}-ext-alb"
  vpc_id = aws_vpc.main.id
}

resource "aws_security_group" "web" {
  name   = "${var.project}-web"
  vpc_id = aws_vpc.main.id
}

resource "aws_security_group" "int_alb" {
  name   = "${var.project}-int-alb"
  vpc_id = aws_vpc.main.id
}

resource "aws_security_group" "app" {
  name   = "${var.project}-app"
  vpc_id = aws_vpc.main.id
}

resource "aws_security_group" "db" {
  name   = "${var.project}-db"
  vpc_id = aws_vpc.main.id
}

resource "aws_security_group" "bastion" {
  name   = "${var.project}-bastion"
  vpc_id = aws_vpc.main.id
}

resource "aws_vpc_security_group_ingress_rule" "ext_alb_http" {
  security_group_id = aws_security_group.ext_alb.id
  cidr_ipv4         = "0.0.0.0/0"
  ip_protocol       = "tcp"
  from_port         = 80
  to_port           = 80
}

resource "aws_vpc_security_group_ingress_rule" "web_from_ext_alb" {
  security_group_id            = aws_security_group.web.id
  referenced_security_group_id = aws_security_group.ext_alb.id
  ip_protocol                  = "tcp"
  from_port                    = 80
  to_port                      = 80
}

resource "aws_vpc_security_group_ingress_rule" "int_alb_from_web" {
  security_group_id            = aws_security_group.int_alb.id
  referenced_security_group_id = aws_security_group.web.id
  ip_protocol                  = "tcp"
  from_port                    = 80
  to_port                      = 80
}

resource "aws_vpc_security_group_ingress_rule" "app_from_int_alb" {
  security_group_id            = aws_security_group.app.id
  referenced_security_group_id = aws_security_group.int_alb.id
  ip_protocol                  = "tcp"
  from_port                    = 8080
  to_port                      = 8080
}

resource "aws_vpc_security_group_ingress_rule" "db_from_app" {
  security_group_id            = aws_security_group.db.id
  referenced_security_group_id = aws_security_group.app.id
  ip_protocol                  = "tcp"
  from_port                    = 3306
  to_port                      = 3306
}

resource "aws_vpc_security_group_ingress_rule" "bastion_ssh" {
  security_group_id = aws_security_group.bastion.id
  cidr_ipv4         = var.my_ip_cidr
  ip_protocol       = "tcp"
  from_port         = 22
  to_port           = 22
}

resource "aws_vpc_security_group_ingress_rule" "web_ssh_from_bastion" {
  security_group_id            = aws_security_group.web.id
  referenced_security_group_id = aws_security_group.bastion.id
  ip_protocol                  = "tcp"
  from_port                    = 22
  to_port                      = 22
}

resource "aws_vpc_security_group_ingress_rule" "app_ssh_from_bastion" {
  security_group_id            = aws_security_group.app.id
  referenced_security_group_id = aws_security_group.bastion.id
  ip_protocol                  = "tcp"
  from_port                    = 22
  to_port                      = 22
}

resource "aws_vpc_security_group_egress_rule" "all" {
  for_each          = toset(["ext_alb", "web", "int_alb", "app", "bastion"])
  security_group_id = local.sg_ids[each.key]
  cidr_ipv4         = "0.0.0.0/0"
  ip_protocol       = "-1"
}

locals {
  sg_ids = {
    ext_alb = aws_security_group.ext_alb.id
    web     = aws_security_group.web.id
    int_alb = aws_security_group.int_alb.id
    app     = aws_security_group.app.id
    bastion = aws_security_group.bastion.id
  }
}

What each line does

• aws_security_group "ext_alb" ... "bastion" — Six groups: one per load balancer, one per tier, one for the bastion.
• ext_alb_http cidr_ipv4 = "0.0.0.0/0" — The only rule open to the internet.
• web_from_ext_alb — Web servers accept port 80 only from the external ALB.
• int_alb_from_web — The internal ALB accepts only the web servers.
• app_from_int_alb port 8080 — App servers accept only the internal ALB.
• db_from_app port 3306 — The database accepts only app servers.
• bastion_ssh cidr_ipv4 = var.my_ip_cidr — SSH to the bastion from your /32 only.
• web_ssh_from_bastion / app_ssh_from_bastion — SSH to private servers only from the bastion.
• aws_vpc_security_group_egress_rule "all" for_each — One outbound rule per group except db, created in a loop.
• locals { sg_ids = {...} } — Maps names to ids so the loop can find each group.

When it is green.

$ terraform validate
Success! The configuration is valid.

When it is red.

Error: Cycle: aws_security_group.web, aws_security_group.app

Fix. A cycle happens when rules are written inline inside two groups that point at each other. Keep the groups empty and put every rule in its own aws_vpc_security_group_ingress_rule resource, as shown.

What: every term in plain English

• security group — A stateful virtual firewall attached to a network interface. It allows traffic; anything not allowed is dropped.
• stateful — If a request is allowed in, the reply is allowed out automatically. You do not write return rules.
• ingress rule — A rule for traffic coming into the resource.
• egress rule — A rule for traffic leaving the resource. AWS adds allow-all egress by default; Terraform removes it unless you write it.
• source security group — A rule that allows traffic from members of another group instead of an IP range. This is how tiers are chained.
• aws_vpc_security_group_ingress_rule — The newer one-rule-per-resource style. Each rule can be changed without touching the others.
• least privilege — Open only the ports and sources a tier really needs, nothing more.
• chain — ext ALB to web, web to internal ALB, internal ALB to app, app to DB. Each link allows only the previous tier.

Example 1: Write the chain as a table first

At Capgemini security reviews the first artefact is a port table. If a row is not in the table, it does not get a rule.

1. List each group.
2. Write its allowed source and port.
3. Turn each row into one rule.

Group     Allows in from          Port
ext_alb   0.0.0.0/0               80
web       ext_alb SG              80
web       bastion SG              22
int_alb   web SG                  80
app       int_alb SG              8080
app       bastion SG              22
db        app SG                  3306
bastion   var.my_ip_cidr          22

Output

8 ingress rules, 6 groups. Only ext_alb is open to the internet, and only on port 80.

What it shows: Notice that no rule uses an IP range except the public ALB and the bastion. Every other link uses a security group as the source.

Example 2: Allow web only from the external ALB

If web servers accepted traffic from anywhere in the VPC, a compromised app server could attack them. Referencing the ALB group keeps the door narrow.

1. Create the web group.
2. Add one ingress rule with referenced_security_group_id.
3. Plan and read the rule.

resource "aws_security_group" "web" {
  name   = "${var.project}-web"
  vpc_id = aws_vpc.main.id
}

resource "aws_vpc_security_group_ingress_rule" "web_from_alb" {
  security_group_id            = aws_security_group.web.id
  referenced_security_group_id = aws_security_group.ext_alb.id
  from_port                    = 80
  to_port                      = 80
  ip_protocol                  = "tcp"
}

Output

# aws_vpc_security_group_ingress_rule.web_from_alb will be created
    + from_port                    = 80
    + referenced_security_group_id = (known after apply)

What it shows: (known after apply) means the ALB group id will be filled in once it exists. Terraform orders the creation for you.

Example 3: Let the database accept only the app tier

Data at HDFC Bank sits behind exactly one rule: port 3306 from the app group. Even the bastion cannot reach the database directly.

1. Create the db group.
2. Allow 3306 from the app group.
3. Check there is no other ingress.

resource "aws_vpc_security_group_ingress_rule" "db_from_app" {
  security_group_id            = aws_security_group.db.id
  referenced_security_group_id = aws_security_group.app.id
  from_port                    = 3306
  to_port                      = 3306
  ip_protocol                  = "tcp"
}

aws ec2 describe-security-groups --group-ids $(terraform output -raw db_sg_id) \
  --query "SecurityGroups[0].IpPermissions[].[FromPort,UserIdGroupPairs[0].GroupId]"

Output

[
    [
        3306,
        "sg-0app1234567890abc"
    ]
]

What it shows: One rule, one source. That is what an auditor wants to see.

Example 4: Add egress rules on purpose

Terraform removes the default allow-all egress. Without an egress rule, servers cannot run dnf install through the NAT and user_data fails quietly.

1. Add an allow-all egress rule for web, app and bastion.
2. Keep the DB group without egress.
3. Plan and count.

resource "aws_vpc_security_group_egress_rule" "all_out" {
  for_each          = { web = aws_security_group.web.id, app = aws_security_group.app.id, bastion = aws_security_group.bastion.id, ext_alb = aws_security_group.ext_alb.id, int_alb = aws_security_group.int_alb.id }
  security_group_id = each.value
  cidr_ipv4         = "0.0.0.0/0"
  ip_protocol       = "-1"
}

Output

# aws_vpc_security_group_egress_rule.all_out["app"] will be created
# aws_vpc_security_group_egress_rule.all_out["bastion"] will be created
# aws_vpc_security_group_egress_rule.all_out["ext_alb"] will be created
# aws_vpc_security_group_egress_rule.all_out["int_alb"] will be created
# aws_vpc_security_group_egress_rule.all_out["web"] will be created

What it shows: ip_protocol = "-1" means all protocols. The DB group needs no egress because replies are stateful.

Example 5: Prove the chain with a port test

A rule on paper is not proof. Teams at Swiggy test each link with nc (netcat) from the previous tier during go-live.

1. SSH to a web server through the bastion.
2. Test the internal ALB on 80.
3. Test the database on 3306 and see it time out.

# on a web server
nc -zv -w 3 $INT_ALB_DNS 80
nc -zv -w 3 $DB_ENDPOINT 3306

Output

Connection to internal-shop3t-int-alb-123.ap-south-1.elb.amazonaws.com 80 port [tcp/http] succeeded!
nc: connect to shop3t-mysql.abc.ap-south-1.rds.amazonaws.com port 3306 (tcp) timed out

What it shows: The second test must fail. Web servers have no business talking to the database; only the app tier can.

Example 6: Find open-to-world rules with a query

Security teams at Infosys run this query weekly. It lists every rule open to 0.0.0.0/0 so you can confirm only the public ALB has one.

1. Query rules with cidr 0.0.0.0/0 in the VPC.
2. Read the group names.
3. Expect only ext_alb ingress.

aws ec2 describe-security-group-rules \
  --filters Name=group-id,Values=$(terraform output -raw ext_alb_sg_id) \
  --query "SecurityGroupRules[?CidrIpv4=='0.0.0.0/0' && !IsEgress].[GroupId,FromPort]"

Output

[
    [
        "sg-0extalb123456789a",
        80
    ]
]

What it shows: Run the same query without the group filter to scan the whole account.

Command cheat sheet

Verify: did it work?

1. Six groups

   aws ec2 describe-security-groups --filters Name=vpc-id,Values=$(terraform output -raw vpc_id) --query "length(SecurityGroups)"
   

   Output

   7 (6 plus the VPC default)
   

2. DB allows only 3306

   terraform state show aws_vpc_security_group_ingress_rule.db_from_app | grep from_port
   

   Output

   from_port = 3306
   

3. Web cannot reach DB

   nc -zv -w 3 $DB_ENDPOINT 3306
   

   Output

   timed out
   

When it is red 1

Error: InvalidPermission.Duplicate: the specified rule already exists

Cause. The same rule is written both inline and as a separate rule resource.

Fix, step by step

1. Use only the separate rule resources.
2. Remove inline ingress blocks.

When it is red 2

dnf: Curl error (28): Timeout was reached for https://cdn.amazonlinux.com

Cause. The server group has no egress rule, so it cannot reach the NAT.

Fix, step by step

1. Add the all_out egress rule.
2. Replace the instance to rerun user_data.

When it is red 3

Error: DependencyViolation: resource sg-... has a dependent object

Cause. A group is still attached to an ENI or used as a source by another rule.

Fix, step by step

1. Let terraform destroy remove dependents first.
2. Delete hand-made ENIs in the console.

Cleanup

terraform destroy -auto-approve

Practice tasks

Homework: try at home

Recap

• Six groups, eight ingress rules.
• Tiers reference groups, not IPs.
• Only the public ALB is open to the world.
• Egress must be written in Terraform.
• Prove each link with nc.

FAQ

Why not use subnet CIDRs as sources?

A group follows the instance wherever it is; a CIDR allows every server in the subnet.

Do I need return rules?

No. Security groups are stateful.

Network ACLs too?

Optional extra layer; groups are enough for this lab.

Why separate rule resources?

Each rule changes independently and plans stay clean.

Interview questions

1. How do you allow only the web tier to reach the app tier?

   Add an ingress rule on the app group with the web or internal ALB group as the source.

2. Are security groups stateful or stateless?

   Stateful; replies are allowed automatically.

3. What happens to egress when Terraform creates a group?

   Terraform removes the default allow-all egress, so you must add it.

4. How would you audit open ports?

   Query rules with 0.0.0.0/0 using describe-security-group-rules or AWS Config.

Samajla ka?

Ravi Sir's Cards for this lesson

Revision cardInterview card

मराठी cards