82.6 Project 90: Policy as code with OPA/Kyverno
Friends, tickets do not stop bad YAML. Today we install Kyverno, require resource limits, and block a Deployment that forgets them.
मित्रांनो, tickets वाईट YAML थांबवत नाहीत. आज Kyverno, resource limits required, आणि विसरलेले Deployment block.
दोस्तों, tickets गलत YAML नहीं रोकते. आज Kyverno, resource limits required, और भूले Deployment को block.
What you will build in this project
- Install Kyverno.
- Write a require-limits policy.
- Apply a bad Deployment and see deny.
- Apply a good Deployment and see allow.
1. Start from a real company story
At CRED in Bengaluru, a memory-hungry Pod without limits starved a noisy node and took payments offline for twelve minutes.
Rough cost: about ₹14 lakh in failed checkouts.
Nisha enforced limits with Kyverno before admission.
Today we write that policy.
Ravindra Bagale's Tip
Students set policies to Audit forever and never Enforce. Start Enforce in non-prod, then prod. Keep that habit.
Students नेहमी Audit ठेवतात Enforce न करता. Non-prod मध्ये Enforce, मग prod. सवय ठेवा.
Students हमेशा Audit रखते हैं Enforce किए बिना. Non-prod में Enforce, फिर prod. आदत रखिए.
2. Start and prepare
Step 1: Why policy as code
Why: Humans miss reviews. We do this before the next step so mistakes stay small and easy to undo. Read the output carefully; the numbers and names tell you if the machine is healthy.
What: echo why. Follow the numbered examples below. Type each command, read the exact output, and match the meaning line before you move on. That is how real teams learn a tool.
Example 1: a1
Type this:
echo 'Admission policies stop bad objects before they land'
You will see:
Admission policies stop bad objects before they land
What it means: Why.
Example 2: a2
Type this:
mkdir -p /tmp/pol90 && echo 'Today: Kyverno, require limits, deny/allow'
You will see:
Today: Kyverno, require limits, deny/allow
What it means: Scope.
Example 3: a3
Type this:
echo 'OPA/Gatekeeper and Kyverno both work; Kyverno YAML is friendlier for many teams'
You will see:
OPA/Gatekeeper and Kyverno both work; Kyverno YAML is friendlier for many teams
What it means: Tools.
Step 2: Install Kyverno
Why: Helm. We do this before the next step so mistakes stay small and easy to undo. Read the output carefully; the numbers and names tell you if the machine is healthy.
What: install. Follow the numbered examples below. Type each command, read the exact output, and match the meaning line before you move on. That is how real teams learn a tool.
Example 4: b1
Type this:
printf '%s\n' 'helm repo add kyverno https://charts.example.com/kyverno/' 'helm install kyverno kyverno/kyverno -n kyverno --create-namespace' > /tmp/pol90/INSTALL.txt && cat /tmp/pol90/INSTALL.txt
You will see:
helm install kyverno kyverno/kyverno -n kyverno --create-namespace
What it means: Install.
Example 5: b2
Type this:
echo 'kubectl -n kyverno get pods'
You will see:
kubectl -n kyverno get pods
What it means: Pods.
Example 6: b3
Type this:
echo 'Wait until the admission controller is Ready'
You will see:
Wait until the admission controller is Ready
What it means: Ready.
Step 3: Require limits
Why: ClusterPolicy. We do this before the next step so mistakes stay small and easy to undo. Read the output carefully; the numbers and names tell you if the machine is healthy.
What: policy. Follow the numbered examples below. Type each command, read the exact output, and match the meaning line before you move on. That is how real teams learn a tool.
Example 7: c1
Type this:
printf '%s\n' 'apiVersion: kyverno.io/v1' 'kind: ClusterPolicy' 'metadata:' ' name: require-limits' 'spec:' ' validationFailureAction: Enforce' ' rules:' ' - name: check-limits' ' match:' ' any:' ' - resources:' ' kinds: [Deployment]' ' validate:' ' message: CPU and memory limits are required' ' pattern:' ' spec:' ' template:' ' spec:' ' containers:' ' - resources:' ' limits:' ' memory: "?*"' ' cpu: "?*"' > /tmp/pol90/policy.yaml && wc -l /tmp/pol90/policy.yaml
You will see:
25 /tmp/pol90/policy.yaml
What it means: Policy.
Example 8: c2
Type this:
echo 'kubectl apply -f /tmp/pol90/policy.yaml'
You will see:
kubectl apply -f /tmp/pol90/policy.yaml
What it means: Apply.
Example 9: c3
Type this:
echo 'Start with validationFailureAction: Audit in prod for a week if needed'
You will see:
Start with validationFailureAction: Audit in prod for a week if needed
What it means: Audit first.
3. Build the main pieces
Step 4: Bad Deployment
Why: Must deny. We do this before the next step so mistakes stay small and easy to undo. Read the output carefully; the numbers and names tell you if the machine is healthy.
What: bad. Follow the numbered examples below. Type each command, read the exact output, and match the meaning line before you move on. That is how real teams learn a tool.
Example 10: d1
Type this:
printf '%s\n' 'apiVersion: apps/v1' 'kind: Deployment' 'metadata: {name: bad-api, namespace: shop}' 'spec:' ' replicas: 1' ' selector: {matchLabels: {app: bad}}' ' template:' ' metadata: {labels: {app: bad}}' ' spec:' ' containers:' ' - name: api' ' image: registry.example.com/shop/api:1' > /tmp/pol90/bad.yaml && echo 'no limits on purpose'
You will see:
no limits on purpose
What it means: Bad.
Example 11: d2
Type this:
echo 'kubectl apply -f /tmp/pol90/bad.yaml'
You will see:
kubectl apply -f /tmp/pol90/bad.yaml
What it means: Apply bad.
Example 12: d3
Type this:
echo 'Expected: rejected by require-limits'
You will see:
Expected: rejected by require-limits
What it means: Expect.
Step 5: Good Deployment
Why: Must allow. We do this before the next step so mistakes stay small and easy to undo. Read the output carefully; the numbers and names tell you if the machine is healthy.
What: good. Follow the numbered examples below. Type each command, read the exact output, and match the meaning line before you move on. That is how real teams learn a tool.
Example 13: e1
Type this:
printf '%s\n' 'apiVersion: apps/v1' 'kind: Deployment' 'metadata: {name: good-api, namespace: shop}' 'spec:' ' replicas: 1' ' selector: {matchLabels: {app: good}}' ' template:' ' metadata: {labels: {app: good}}' ' spec:' ' containers:' ' - name: api' ' image: registry.example.com/shop/api:1' ' resources:' ' limits: {cpu: 250m, memory: 256Mi}' ' requests: {cpu: 50m, memory: 64Mi}' > /tmp/pol90/good.yaml && wc -l /tmp/pol90/good.yaml
You will see:
16 /tmp/pol90/good.yaml
What it means: Good.
Example 14: e2
Type this:
echo 'kubectl apply -f /tmp/pol90/good.yaml'
You will see:
kubectl apply -f /tmp/pol90/good.yaml
What it means: Apply good.
Example 15: e3
Type this:
printf '%s\n' 'Acceptance: bad denied, good created' > /tmp/pol90/ACCEPT.txt && cat /tmp/pol90/ACCEPT.txt
You will see:
Acceptance: bad denied, good created
What it means: Gate.
Step 6: Exceptions
Why: Break glass. We do this before the next step so mistakes stay small and easy to undo. Read the output carefully; the numbers and names tell you if the machine is healthy.
What: exc. Follow the numbered examples below. Type each command, read the exact output, and match the meaning line before you move on. That is how real teams learn a tool.
Example 16: f1
Type this:
echo 'PolicyException can allow a named Deployment for a limited time'
You will see:
PolicyException can allow a named Deployment for a limited time
What it means: Exception.
Example 17: f2
Type this:
echo 'Require a ticket annotation on every exception'
You will see:
Require a ticket annotation on every exception
What it means: Ticket.
Example 18: f3
Type this:
echo 'Review exceptions weekly; delete expired ones'
You will see:
Review exceptions weekly; delete expired ones
What it means: Review.
4. Automate and verify
Step 7: More policies
Why: Next rules. We do this before the next step so mistakes stay small and easy to undo. Read the output carefully; the numbers and names tell you if the machine is healthy.
What: more. Follow the numbered examples below. Type each command, read the exact output, and match the meaning line before you move on. That is how real teams learn a tool.
Example 19: g1
Type this:
printf '%s\n' 'Disallow latest tag' 'Require non-root' 'Require readOnlyRootFilesystem where possible' 'Restrict hostNetwork' > /tmp/pol90/MORE.txt && cat /tmp/pol90/MORE.txt
You will see:
Disallow latest tag
What it means: More.
Example 20: g2
Type this:
echo 'Add one policy at a time; measure deny rates'
You will see:
Add one policy at a time; measure deny rates
What it means: Pace.
Example 21: g3
Type this:
echo 'Generate reports from PolicyReports CRDs'
You will see:
Generate reports from PolicyReports CRDs
What it means: Reports.
Step 8: OPA note
Why: Same idea. We do this before the next step so mistakes stay small and easy to undo. Read the output carefully; the numbers and names tell you if the machine is healthy.
What: opa. Follow the numbered examples below. Type each command, read the exact output, and match the meaning line before you move on. That is how real teams learn a tool.
Example 22: h1
Type this:
echo 'Gatekeeper uses ConstraintTemplates and Constraints'
You will see:
Gatekeeper uses ConstraintTemplates and Constraints
What it means: Gatekeeper.
Example 23: h2
Type this:
echo 'Rego is powerful; Kyverno patterns are enough for many teams'
You will see:
Rego is powerful; Kyverno patterns are enough for many teams
What it means: Rego.
Example 24: h3
Type this:
echo 'Pick one engine per cluster to avoid double admits'
You will see:
Pick one engine per cluster to avoid double admits
What it means: One engine.
Step 9: Rollout
Why: Audit then Enforce. We do this before the next step so mistakes stay small and easy to undo. Read the output carefully; the numbers and names tell you if the machine is healthy.
What: rollout. Follow the numbered examples below. Type each command, read the exact output, and match the meaning line before you move on. That is how real teams learn a tool.
Example 25: i1
Type this:
printf '%s\n' 'Week 1: Audit in prod, Enforce in stage' 'Week 2: Fix top offenders' 'Week 3: Enforce in prod' > /tmp/pol90/ROLL.txt && cat /tmp/pol90/ROLL.txt
You will see:
Week 3: Enforce in prod
What it means: Rollout.
Example 26: i2
Type this:
echo 'Announce the deny message so developers know how to fix'
You will see:
Announce the deny message so developers know how to fix
What it means: Announce.
Example 27: i3
Type this:
echo 'Watch admission latency after install'
You will see:
Watch admission latency after install
What it means: Latency.
5. Harden and accept
Step 10: Story CRED
Why: No limits. We do this before the next step so mistakes stay small and easy to undo. Read the output carefully; the numbers and names tell you if the machine is healthy.
What: story. Follow the numbered examples below. Type each command, read the exact output, and match the meaning line before you move on. That is how real teams learn a tool.
Example 28: j1
Type this:
printf '%s\n' 'At CRED in Bengaluru, a Pod without limits starved a node for twelve minutes.' 'Nisha enforced limits with Kyverno at admission.' > /tmp/pol90/STORY.txt && wc -l /tmp/pol90/STORY.txt
You will see:
2 /tmp/pol90/STORY.txt
What it means: Story.
Example 29: j2
Type this:
echo 'Rough cost: about ₹14 lakh in failed checkouts'
You will see:
Rough cost: about ₹14 lakh in failed checkouts
What it means: Cost.
Example 30: j3
Type this:
echo 'Nisha: PASS'
You will see:
Nisha: PASS
What it means: Sign-off.
Step 11: Proof
Why: Files. We do this before the next step so mistakes stay small and easy to undo. Read the output carefully; the numbers and names tell you if the machine is healthy.
What: ls. Follow the numbered examples below. Type each command, read the exact output, and match the meaning line before you move on. That is how real teams learn a tool.
Example 31: k1
Type this:
ls /tmp/pol90
You will see:
ACCEPT.txt\nINSTALL.txt\nMORE.txt\nROLL.txt\nSTORY.txt\nbad.yaml\ngood.yaml\npolicy.yaml
What it means: Tree.
Example 32: k2
Type this:
grep Enforce /tmp/pol90/policy.yaml
You will see:
validationFailureAction: Enforce
What it means: Enforce.
Example 33: k3
Type this:
echo 'GitOps/DevSecOps projects 85-90 complete'
You will see:
GitOps/DevSecOps projects 85-90 complete
What it means: Done.
Step 12: Acceptance
Why: Bad denied, good allowed. We do this before the next step so mistakes stay small and easy to undo. Read the output carefully; the numbers and names tell you if the machine is healthy.
What: final. Follow the numbered examples below. Type each command, read the exact output, and match the meaning line before you move on. That is how real teams learn a tool.
Example 34: z1
Type this:
echo 'Kyverno installed'
You will see:
Kyverno installed
What it means: Install.
Example 35: z2
Type this:
echo 'Bad Deployment denied'
You will see:
Bad Deployment denied
What it means: Deny.
Example 36: z3
Type this:
echo 'Good Deployment allowed'
You will see:
Good Deployment allowed
What it means: Allow.
Check yourself
Q1. Who fixed CRED OOM?
Nisha
Q2. Tool?
Kyverno
Q3. Mode first?
Audit
Q4. Missing limits?
deny
Q5. Good YAML?
allow
Q6. Tickets enough?
no
Friends, you blocked bad YAML at admission. Did you understand? Let us try it once more. Flip the policy to Audit, apply bad YAML, then Enforce again and prove deny returns.
मित्रांनो, तुम्ही admission वर वाईट YAML block केले. Policy Audit करा, bad YAML apply करा, मग Enforce करून deny येते हे सिद्ध करा.
दोस्तों, आपने admission पर गलत YAML block किया. Policy Audit करें, bad YAML apply करें, फिर Enforce कर deny आए, यह साबित करें.
Lab: try at home
- Install Kyverno.
- Require-limits policy.
- Bad Deployment denied.
- Good Deployment allowed.
Recap
- Policy as code.
- Admit before etcd.
- Enforce after Audit trial.
- Limits on every container.
- Exceptions via policy exceptions.
Lab
- Finish all six projects; keep ACCEPT notes for each.
- Never commit Vault root tokens, cloud keys, or synced Secret YAML.
- Enforce policies after an Audit trial; do not skip Enforce forever.
- After each project, write one paragraph: what broke and how you fixed it.
- Next batch starts Monitoring projects 91-96.
Practice task
Pick the Argo sync and the Kyverno deny. Prove both before you call the chapter done.
Ravindra Bagale's Tip
Git is desired state. Promote with PRs. Secrets leave git. Sync from a store. Scan before merge. Enforce after Audit.
Git = desired state. PR ने promote. Secrets git बाहेर. Store वरून sync. Merge आधी scan. Audit नंतर Enforce.
Git = desired state. PR से promote. Secrets git के बाहर. Store से sync. Merge से पहले scan. Audit के बाद Enforce.
In short
- Git drives the cluster.
- Overlays per env.
- Vault for secrets.
- External Secrets sync.
- Scan on every PR.
- Admit with Kyverno.