Chapter 78: DevOps Projects Part 8: Terraform Three-Tier EKS CI Drift CloudFormation Infracost
78.3 Project 57: Terraform plan and apply in a CI pipeline
Friends, laptop apply does not scale. Today we write a CI shape: fmt and validate on every push, plan on pull request with a saved plan artifact, and apply on main only after approval — without putting secrets in the pipeline YAML.
मित्रांनो, laptop apply scale होत नाही. आज CI shape: push वर fmt/validate, PR वर plan artifact, main वर approval नंतर apply — secrets YAML मध्ये नाही.
दोस्तों, laptop apply scale नहीं होता. आज CI shape: push पर fmt/validate, PR पर plan artifact, main पर approval बाद apply — secrets YAML में नहीं.
What you will build in this project
- fmt validate script.
- plan on PR notes.
- apply on main with approval.
- OIDC or role notes — no long-lived keys in YAML.
1. Start from a real company story
At Razorpay in Bengaluru, a Friday laptop apply to prod used stale state and removed a security group rule.
Payments from one partner failed for an hour. Rough cost: about ₹70 lakh in volume and a postmortem.
Isha moved plan to pull requests and apply to main with a human approval gate.
Today we write that CI loop.
Ravindra Bagale's Tip
Students paste AWS keys into CI variables and leave them forever. Prefer OIDC short-lived roles. Keep that habit.
Students CI variables मध्ये AWS keys paste करतात. OIDC short-lived roles वापरा. सवय ठेवा.
Students CI variables में AWS keys paste करते हैं. OIDC short-lived roles इस्तेमाल करें. आदत रखिए.
2. Start and prepare
Step 1: Why CI TF
Why: Same commands every time. We do this before the next step so mistakes stay small and easy to undo. Read the output carefully; the numbers and names tell you if the machine is healthy.
What: echo why. Follow the numbered examples below. Type each command, read the exact output, and match the meaning line before you move on. That is how real teams learn a tool.
Example 1: a1
Type this:
echo 'CI runs fmt validate plan apply so laptops stop being snowflakes'
You will see:
CI runs fmt validate plan apply so laptops stop being snowflakes
What it means: Why.
Example 2: a2
Type this:
mkdir -p /tmp/tf57/scripts && printf '%s\n' 'Rule: plan on PR, apply on main after approval' > /tmp/tf57/RULE.txt && cat /tmp/tf57/RULE.txt
You will see:
Rule: plan on PR, apply on main after approval
What it means: Rule.
Example 3: a3
Type this:
echo 'Never apply from a feature branch to prod state'
You will see:
Never apply from a feature branch to prod state
What it means: Branch rule.
Step 2: fmt validate
Why: Every push. We do this before the next step so mistakes stay small and easy to undo. Read the output carefully; the numbers and names tell you if the machine is healthy.
What: scripts/fmt-validate.sh. Follow the numbered examples below. Type each command, read the exact output, and match the meaning line before you move on. That is how real teams learn a tool.
Example 4: b1
Type this:
printf '%s\n' '#!/bin/bash' 'set -euo pipefail' 'terraform fmt -check -recursive' 'terraform init -backend=false -input=false' 'terraform validate' > /tmp/tf57/scripts/fmt-validate.sh && chmod 755 /tmp/tf57/scripts/fmt-validate.sh && cat /tmp/tf57/scripts/fmt-validate.sh
You will see:
terraform fmt -check -recursive
What it means: Fmt and validate script.
Example 5: b2
Type this:
echo 'fmt -check fails the job if files need formatting'
You will see:
fmt -check fails the job if files need formatting
What it means: Check mode.
Example 6: b3
Type this:
echo 'init -backend=false is enough for validate without credentials'
You will see:
init -backend=false is enough for validate without credentials
What it means: Validate tip.
Step 3: plan script
Why: Save a plan file. We do this before the next step so mistakes stay small and easy to undo. Read the output carefully; the numbers and names tell you if the machine is healthy.
What: scripts/plan.sh. Follow the numbered examples below. Type each command, read the exact output, and match the meaning line before you move on. That is how real teams learn a tool.
Example 7: c1
Type this:
printf '%s\n' '#!/bin/bash' 'set -euo pipefail' 'terraform init -input=false' 'terraform plan -input=false -out=tfplan' 'terraform show -no-color tfplan > plan.txt' > /tmp/tf57/scripts/plan.sh && chmod 755 /tmp/tf57/scripts/plan.sh && cat /tmp/tf57/scripts/plan.sh
You will see:
terraform plan -input=false -out=tfplan
What it means: Plan artifact.
Example 8: c2
Type this:
echo 'Upload tfplan and plan.txt as CI artifacts on the pull request'
You will see:
Upload tfplan and plan.txt as CI artifacts on the pull request
What it means: Artifact tip.
Example 9: c3
Type this:
echo 'Reviewers read plan.txt before approval'
You will see:
Reviewers read plan.txt before approval
What it means: Review tip.
3. Build the main pieces
Step 4: apply script
Why: Apply the reviewed file. We do this before the next step so mistakes stay small and easy to undo. Read the output carefully; the numbers and names tell you if the machine is healthy.
What: scripts/apply.sh. Follow the numbered examples below. Type each command, read the exact output, and match the meaning line before you move on. That is how real teams learn a tool.
Example 10: d1
Type this:
printf '%s\n' '#!/bin/bash' 'set -euo pipefail' 'terraform init -input=false' 'terraform apply -input=false tfplan' > /tmp/tf57/scripts/apply.sh && chmod 755 /tmp/tf57/scripts/apply.sh && cat /tmp/tf57/scripts/apply.sh
You will see:
terraform apply -input=false tfplan
What it means: Apply saved plan.
Example 11: d2
Type this:
echo 'Applying tfplan refuses unexpected changes that appeared after review'
You will see:
Applying tfplan refuses unexpected changes that appeared after review
What it means: Why -out.
Example 12: d3
Type this:
echo 'If the plan is stale, re-run plan — do not apply -auto-approve blindly'
You will see:
If the plan is stale, re-run plan — do not apply -auto-approve blindly
What it means: Stale tip.
Step 5: Pipeline sketch
Why: Stages without a banned host. We do this before the next step so mistakes stay small and easy to undo. Read the output carefully; the numbers and names tell you if the machine is healthy.
What: ci-NOTES.txt. Follow the numbered examples below. Type each command, read the exact output, and match the meaning line before you move on. That is how real teams learn a tool.
Example 13: e1
Type this:
printf '%s\n' 'CI stages:' '1) on push: fmt-validate.sh' '2) on pull_request: plan.sh + upload artifacts' '3) on main: manual approval + apply.sh' > /tmp/tf57/ci-NOTES.txt && cat /tmp/tf57/ci-NOTES.txt
You will see:
3) on main: manual approval + apply.sh
What it means: Stages.
Example 14: e2
Type this:
echo 'Use your org CI product — Jenkins, GitLab CI, or Actions — same stages'
You will see:
Use your org CI product — Jenkins, GitLab CI, or Actions — same stages
What it means: Tool agnostic.
Example 15: e3
Type this:
echo 'Protect the main branch so apply cannot skip review'
You will see:
Protect the main branch so apply cannot skip review
What it means: Branch protection.
Step 6: OIDC notes
Why: Short-lived cloud creds. We do this before the next step so mistakes stay small and easy to undo. Read the output carefully; the numbers and names tell you if the machine is healthy.
What: OIDC-NOTES.txt. Follow the numbered examples below. Type each command, read the exact output, and match the meaning line before you move on. That is how real teams learn a tool.
Example 16: f1
Type this:
printf '%s\n' 'Auth:' '- prefer cloud OIDC from the CI identity' '- IAM role trust the CI issuer' '- no long-lived Access Key Id in pipeline YAML' > /tmp/tf57/OIDC-NOTES.txt && cat /tmp/tf57/OIDC-NOTES.txt
You will see:
- no long-lived Access Key Id in pipeline YAML
What it means: OIDC.
Example 17: f2
Type this:
echo 'Separate roles for plan-only and apply'
You will see:
Separate roles for plan-only and apply
What it means: Role split.
Example 18: f3
Type this:
echo 'Plan role needs read; apply role needs write on the target account'
You will see:
Plan role needs read; apply role needs write on the target account
What it means: Permissions.
4. Automate and verify
Step 7: Backend in CI
Why: Same remote state. We do this before the next step so mistakes stay small and easy to undo. Read the output carefully; the numbers and names tell you if the machine is healthy.
What: backend tip. Follow the numbered examples below. Type each command, read the exact output, and match the meaning line before you move on. That is how real teams learn a tool.
Example 19: g1
Type this:
echo 'CI must use the same S3 backend and lock table as humans'
You will see:
CI must use the same S3 backend and lock table as humans
What it means: Backend sameness.
Example 20: g2
Type this:
echo 'Partial backend config can come from -backend-config=env.hcl'
You will see:
Partial backend config can come from -backend-config=env.hcl
What it means: Backend config.
Example 21: g3
Type this:
echo 'Locks still work — two pipelines will wait instead of corrupt state'
You will see:
Locks still work — two pipelines will wait instead of corrupt state
What it means: Lock tip.
Step 8: Var files in CI
Why: dev stage prod. We do this before the next step so mistakes stay small and easy to undo. Read the output carefully; the numbers and names tell you if the machine is healthy.
What: var-file tip. Follow the numbered examples below. Type each command, read the exact output, and match the meaning line before you move on. That is how real teams learn a tool.
Example 22: h1
Type this:
echo 'terraform plan -var-file=stage.tfvars -out=tfplan'
You will see:
terraform plan -var-file=stage.tfvars -out=tfplan
What it means: Var file.
Example 23: h2
Type this:
echo 'Prod apply uses prod.tfvars and a stricter approval'
You will see:
Prod apply uses prod.tfvars and a stricter approval
What it means: Prod gate.
Example 24: h3
Type this:
echo 'Never mix stage state with prod var-file'
You will see:
Never mix stage state with prod var-file
What it means: Mismatch warning.
Step 9: Policy checks
Why: Optional gate. We do this before the next step so mistakes stay small and easy to undo. Read the output carefully; the numbers and names tell you if the machine is healthy.
What: policy notes. Follow the numbered examples below. Type each command, read the exact output, and match the meaning line before you move on. That is how real teams learn a tool.
Example 25: i1
Type this:
printf '%s\n' 'Optional: run tfsec or Checkov on the plan JSON before approval' > /tmp/tf57/POLICY-NOTES.txt && cat /tmp/tf57/POLICY-NOTES.txt
You will see:
Optional: run tfsec or Checkov on the plan JSON before approval
What it means: Policy tip.
Example 26: i2
Type this:
echo 'Fail the PR when a public RDS or open SG appears in the plan'
You will see:
Fail the PR when a public RDS or open SG appears in the plan
What it means: Example policy.
Example 27: i3
Type this:
echo 'Policy as code comes back in the DevSecOps projects'
You will see:
Policy as code comes back in the DevSecOps projects
What it means: Bridge.
5. Harden and accept
Step 10: Smoke test scripts
Why: Prove locally. We do this before the next step so mistakes stay small and easy to undo. Read the output carefully; the numbers and names tell you if the machine is healthy.
What: run fmt in a tiny module. Follow the numbered examples below. Type each command, read the exact output, and match the meaning line before you move on. That is how real teams learn a tool.
Example 28: j1
Type this:
mkdir -p /tmp/tf57/sample && printf '%s\n' 'terraform { required_version = ">= 1.5.0" }' > /tmp/tf57/sample/main.tf && cd /tmp/tf57/sample && terraform fmt -check && echo fmt-ok
You will see:
fmt-ok
What it means: Fmt passes on sample.
Example 29: j2
Type this:
cd /tmp/tf57/sample && terraform init -backend=false -input=false >/tmp/tf57/v.log 2>&1 && terraform validate && echo validate-ok
You will see:
validate-ok
What it means: Validate passes.
Example 30: j3
Type this:
ls /tmp/tf57/scripts
You will see:
fmt-validate.sh\nplan.sh\napply.sh
What it means: Scripts present.
Step 11: Acceptance
Why: Isha signs off. We do this before the next step so mistakes stay small and easy to undo. Read the output carefully; the numbers and names tell you if the machine is healthy.
What: final. Follow the numbered examples below. Type each command, read the exact output, and match the meaning line before you move on. That is how real teams learn a tool.
Example 31: k1
Type this:
cat /tmp/tf57/RULE.txt
You will see:
Rule: plan on PR, apply on main after approval
What it means: Rule kept.
Example 32: k2
Type this:
echo 'OIDC notes present; no Access Key Id in scripts'
You will see:
OIDC notes present; no Access Key Id in scripts
What it means: Secret check.
Example 33: k3
Type this:
echo 'Isha: PASS'
You will see:
Isha: PASS
What it means: Accepted.
Step 12: Recap bridge
Why: Next drift. We do this before the next step so mistakes stay small and easy to undo. Read the output carefully; the numbers and names tell you if the machine is healthy.
What: bridge. Follow the numbered examples below. Type each command, read the exact output, and match the meaning line before you move on. That is how real teams learn a tool.
Example 34: z1
Type this:
echo 'CI without a saved plan is just remote auto-approve'
You will see:
CI without a saved plan is just remote auto-approve
What it means: Warning.
Example 35: z2
Type this:
echo 'Keep plan.txt next to the approval ticket'
You will see:
Keep plan.txt next to the approval ticket
What it means: Habit.
Example 36: z3
Type this:
echo 'Next: drift detection and import — project 58'
You will see:
Next: drift detection and import — project 58
What it means: Bridge.
Check yourself
Q1. Who fixed Razorpay laptop apply?
Isha
Q2. PR runs?
plan
Q3. Main runs?
apply
Q4. Saved plan flag?
-out
Q5. Static keys in YAML?
no
Q6. Format command?
terraform fmt
Friends, you designed Terraform CI with plan on PR and apply on main. Did you understand? Let us try it once more. Add a drift cron that posts plan output to Slack.
मित्रांनो, तुम्ही PR वर plan आणि main वर apply असे Terraform CI डिझाइन केले. Slack वर drift cron टाका.
दोस्तों, आपने PR पर plan और main पर apply वाला Terraform CI डिज़ाइन किया. Slack पर drift cron डालें.
Lab: try at home
- Write fmt validate script.
- Write plan script with -out.
- Write apply script that uses the plan file.
- Document OIDC.
- Acceptance.
Recap
- CI runs the same commands as you.
- Plan artifacts freeze the change.
- Apply the file you reviewed.
- OIDC beats static keys.
- Main is the only apply branch.