Ravindra BagaleCourses & study guides मराठी

Chapter 60: Kubernetes Workload Controllers in Depth: ReplicaSet, Deployment, StatefulSet, DaemonSet, Jobs and CronJobs

60.3 Deployment internals: revisions, strategy, maxSurge and maxUnavailable

Why: the problem

Suppose Swiggy releases a new version of its order API at 7 PM, the busiest dinner hour. It runs 10 pods. If all 10 are replaced at once, there is a gap with no pods and orders fail. If they are replaced one by one too slowly, the release takes an hour. A Deployment solves this. It does not replace pods itself. It creates a new ReplicaSet for the new version and moves pods from the old ReplicaSet to the new one, step by step. Two numbers control the speed and safety: maxSurge, how many extra pods may exist, and maxUnavailable, how many pods may be missing. Every change to the pod template makes a new revision. Old ReplicaSets are kept at 0 replicas, so you can roll back in seconds. In this lesson we look inside a rollout: the ReplicaSets, the revision numbers, the annotations, the Recreate strategy, pause and resume, and progressDeadlineSeconds. Then we break it with a bad image and with settings that allow no progress. Lesson 18 showed the basics; here we see the machinery.

What: every term in plain English

  • Deployment — An apps/v1 object that manages ReplicaSets to roll out and roll back pod template changes.
  • Revision — A number stored in the annotation deployment.kubernetes.io/revision on each ReplicaSet. It goes up with every template change.
  • strategy.type — RollingUpdate (default) or Recreate.
  • maxSurge — How many pods above replicas may exist during a rollout. Number or percent; default 25%.
  • maxUnavailable — How many pods below replicas may be not ready. Default 25%. Both cannot be 0.
  • minReadySeconds — A new pod must stay Ready this long before it counts as available.
  • progressDeadlineSeconds — If no progress for this long (default 600), the Deployment reports ProgressDeadlineExceeded.
  • revisionHistoryLimit — How many old ReplicaSets to keep. Default 10.
  • Paused — While paused, template changes do not start a rollout. Useful for batching changes.
  • change-cause — The annotation kubernetes.io/change-cause, shown in rollout history.
Set replicas and labels. Pick a strategy. Tune surge and unavailable. Add readiness and minReadySeconds. Record a change-cause. Watch the rollout. Undo if needed.
Rolling update: new ReplicaSet up, old ReplicaSet down

How: step by step

  1. Set replicas and labels. Same as before; selector is immutable.
  2. Pick a strategy. RollingUpdate for web apps; Recreate when two versions must never run together.
  3. Tune surge and unavailable. Zero downtime: maxUnavailable 0, maxSurge 1 or 25%.
  4. Add readiness and minReadySeconds. A rollout is only as safe as your readiness probe.
  5. Record a change-cause. kubectl annotate after each release.
  6. Watch the rollout. kubectl rollout status and kubectl get rs -w.
  7. Undo if needed. kubectl rollout undo, optionally --to-revision.

Example 1: Zero-downtime rolling update with maxSurge 1 and maxUnavailable 0

Goal. Zero-downtime rolling update with maxSurge 1 and maxUnavailable 0.

orders-deploy.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: orders
  namespace: shop
  annotations:
    kubernetes.io/change-cause: "v1 nginx 1.26"
spec:
  replicas: 4
  revisionHistoryLimit: 5
  minReadySeconds: 5
  progressDeadlineSeconds: 120
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  selector:
    matchLabels:
      app: orders
  template:
    metadata:
      labels:
        app: orders
    spec:
      containers:
        - name: web
          image: nginx:1.26
          ports:
            - containerPort: 80
          readinessProbe:
            httpGet:
              path: /
              port: 80
            periodSeconds: 2
          resources:
            requests: {cpu: 20m, memory: 32Mi}
            limits: {memory: 64Mi}

What each line does

  • replicas: 4 — Normal size.
  • maxSurge: 1 — At most 5 pods during the rollout.
  • maxUnavailable: 0 — Never fewer than 4 ready pods. Capacity never drops.
  • minReadySeconds: 5 — A new pod must be Ready for 5 seconds before the next old pod goes.
  • progressDeadlineSeconds: 120 — Report failure after 2 minutes with no progress.
  • revisionHistoryLimit: 5 — Keep 5 old ReplicaSets for rollback.
  • readinessProbe — Without it, a pod is Ready as soon as it starts, even if the app is not.

Green output

$ kubectl apply -f orders-deploy.yaml
deployment.apps/orders created
$ kubectl set image deploy/orders -n shop web=nginx:1.27
$ kubectl annotate deploy/orders -n shop kubernetes.io/change-cause="v2 nginx 1.27" --overwrite
$ kubectl get rs -n shop -l app=orders -w
NAME                DESIRED   CURRENT   READY   AGE
orders-6b7f9c5d8    4         4         4       1m
orders-84c6d7f5b    1         1         0       0s
orders-84c6d7f5b    1         1         1       2s
orders-6b7f9c5d8    3         4         4       7s
orders-84c6d7f5b    2         2         1       7s
...
orders-84c6d7f5b    4         4         4       30s
orders-6b7f9c5d8    0         0         0       1m30s
$ kubectl rollout history deploy/orders -n shop
REVISION  CHANGE-CAUSE
1         v1 nginx 1.26
2         v2 nginx 1.27

Expected result. One new pod is added, waits 5 seconds Ready, then one old pod is removed. Ready pods never went below 4.

Example 2: Recreate strategy for an app that cannot run two versions

Goal. Recreate strategy for an app that cannot run two versions.

billing-deploy.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: billing
  namespace: shop
spec:
  replicas: 2
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: billing
  template:
    metadata:
      labels:
        app: billing
    spec:
      containers:
        - name: worker
          image: busybox:1.36
          command: ["sh", "-c", "echo billing $VERSION started; sleep 3600"]
          env:
            - name: VERSION
              value: "v1"

What each line does

  • type: Recreate — Scale old RS to 0 first, then start the new RS.
  • No rollingUpdate block — It is not allowed with Recreate.
  • When to use — A database schema change, a single-writer job, or a licence that allows one version only.
  • Cost — There is a short downtime between old pods ending and new pods starting.

Green output

$ kubectl apply -f billing-deploy.yaml
$ kubectl set env deploy/billing -n shop VERSION=v2
$ kubectl get pods -n shop -l app=billing -w
billing-7d4f8b9c6-2kq9x   1/1   Terminating         0   40s
billing-7d4f8b9c6-v8s2m   1/1   Terminating         0   40s
billing-5c9b6f7d4-hn4tr   0/1   Pending             0   0s
billing-5c9b6f7d4-hn4tr   0/1   ContainerCreating   0   0s
billing-5c9b6f7d4-x2l7p   1/1   Running             0   3s
$ kubectl logs -n shop deploy/billing
billing v2 started

Expected result. No new pod started until every old pod was gone. v1 and v2 never ran together.

Example 3: Pause, batch changes, resume, and catch a stuck rollout

Goal. Pause, batch changes, resume, and catch a stuck rollout.

orders-bad.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: orders
  namespace: shop
spec:
  replicas: 4
  progressDeadlineSeconds: 60
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  selector:
    matchLabels:
      app: orders
  template:
    metadata:
      labels:
        app: orders
    spec:
      containers:
        - name: web
          image: nginx:9.9-does-not-exist
          readinessProbe:
            httpGet: {path: /, port: 80}

What each line does

  • pause — kubectl rollout pause stops new rollouts while you make many changes.
  • resume — All changes roll out as one revision.
  • Bad image — The new pod never becomes Ready.
  • maxUnavailable 0 saves you — Old pods keep serving; users see no outage.
  • progressDeadlineSeconds: 60 — After 60 seconds the condition Progressing=False with reason ProgressDeadlineExceeded.

Green output

$ kubectl rollout pause deploy/orders -n shop
$ kubectl set image deploy/orders -n shop web=nginx:1.27-alpine
$ kubectl set resources deploy/orders -n shop -c web --limits=memory=96Mi
$ kubectl rollout resume deploy/orders -n shop
$ kubectl rollout history deploy/orders -n shop | tail -1
3         <none>
$ kubectl apply -f orders-bad.yaml
$ kubectl rollout status deploy/orders -n shop --timeout=90s
Waiting for deployment "orders" rollout to finish: 1 out of 4 new replicas have been updated...
error: deployment "orders" exceeded its progress deadline
$ kubectl get pods -n shop -l app=orders
NAME                      READY   STATUS             RESTARTS   AGE
orders-7f5d9c8b6-2mlkp    1/1     Running            0          3m
orders-7f5d9c8b6-9wq4z    1/1     Running            0          3m
orders-7f5d9c8b6-k8vbn    1/1     Running            0          3m
orders-7f5d9c8b6-tx6rs    1/1     Running            0          3m
orders-c8f6b4d9-q7h2c     0/1     ImagePullBackOff   0          70s
$ kubectl rollout undo deploy/orders -n shop
deployment.apps/orders rolled back

Expected result. Two changes became one revision. The bad image stopped at one pod, 4 old pods kept serving, and undo cleaned up. In CI, rollout status exits non-zero, so the pipeline fails.

Every command, step by step (kind lab with kubectl)

  1. See the chain

    kubectl get deploy,rs,pods -n shop -l app=orders
    

    What it does: Deployment, then ReplicaSets, then Pods.

    deployment.apps/orders   4/4   4   4
    replicaset.apps/orders-84c6d7f5b   4   4   4
    replicaset.apps/orders-6b7f9c5d8   0   0   0
    
  2. Read the revision annotation

    kubectl get rs -n shop -l app=orders -o custom-columns=NAME:.metadata.name,REV:.metadata.annotations.deployment\.kubernetes\.io/revision,DESIRED:.spec.replicas
    

    What it does: Each RS knows its revision number.

    NAME               REV   DESIRED
    orders-6b7f9c5d8   1     0
    orders-84c6d7f5b   2     4
    
  3. Read the strategy

    kubectl get deploy orders -n shop -o jsonpath='{.spec.strategy}{"\n"}'
    

    What it does: Confirms surge and unavailable.

    {"rollingUpdate":{"maxSurge":1,"maxUnavailable":0},"type":"RollingUpdate"}
    
  4. Detail of one revision

    kubectl rollout history deploy/orders -n shop --revision=1
    

    What it does: Shows the full pod template of that revision.

    deployment.apps/orders with revision #1
      Containers:
       web:
        Image:      nginx:1.26
    
  5. Read conditions

    kubectl get deploy orders -n shop -o jsonpath='{range .status.conditions[*]}{.type}={.status} {.reason}{"\n"}{end}'
    

    What it does: Available and Progressing tell you the health of the rollout.

    Available=True MinimumReplicasAvailable
    Progressing=True NewReplicaSetAvailable
    
  6. Restart without a change

    kubectl rollout restart deploy/orders -n shop
    

    What it does: Adds a restartedAt annotation, so a new RS rolls all pods. Good after a ConfigMap change.

    deployment.apps/orders restarted
    
  7. Roll back to a revision

    kubectl rollout undo deploy/orders -n shop --to-revision=1
    kubectl get deploy orders -n shop -o jsonpath='{.spec.template.spec.containers[0].image}{"\n"}'
    

    What it does: Old RS is scaled up again; it gets a new revision number.

    nginx:1.26
    
  8. Scale is not a revision

    kubectl scale deploy/orders -n shop --replicas=6
    kubectl rollout history deploy/orders -n shop | tail -1
    

    What it does: Only template changes make revisions.

    deployment.apps/orders scaled
    
  9. Clean up

    kubectl delete deploy orders billing -n shop
    

    What it does: Ready for StatefulSets.

    deployment.apps "orders" deleted
    deployment.apps "billing" deleted
    

When it is red 1

The Deployment "orders" is invalid: spec.strategy.rollingUpdate.maxUnavailable: Invalid value: intstr.IntOrString{Type:0, IntVal:0}: may not be 0 when `maxSurge` is 0

Cause. Both values are 0, so the rollout could never move.

Fix, step by step

  1. Set maxSurge to 1 or 25%.
  2. Keep maxUnavailable 0 for zero downtime.

When it is red 2

error: deployment "orders" exceeded its progress deadline

Cause. New pods are not getting Ready: bad image, failing probe, or no capacity.

Fix, step by step

  1. kubectl get pods and describe the new pod.
  2. Fix the image or probe, or run kubectl rollout undo.
  3. Check node capacity if pods are Pending.

When it is red 3

spec.strategy.rollingUpdate: Forbidden: may not be specified when strategy `type` is 'Recreate'

Cause. A rollingUpdate block is left over under Recreate.

Fix, step by step

  1. Delete the rollingUpdate block.

When it is red 4

rollout history shows CHANGE-CAUSE <none>

Cause. No change-cause annotation; --record is deprecated.

Fix, step by step

  1. After each release run kubectl annotate deploy/NAME kubernetes.io/change-cause="text" --overwrite.
  2. Or set it in the YAML metadata from your CI pipeline.

Practice tasks

Easy Task 1: Apply

Why: Basic rollout.

What to do: Apply orders-deploy.yaml and check 4/4.

How:
  1. kubectl get deploy.
Solution
kubectl apply -f orders-deploy.yaml

Easy Task 2: Update

Why: New revision.

What to do: Set the image to nginx:1.27 and watch ReplicaSets.

How:
  1. get rs -w.
Solution
kubectl set image deploy/orders -n shop web=nginx:1.27

Easy Task 3: History

Why: Read revisions.

What to do: Show the history with change-cause.

How:
  1. annotate first.
Solution
kubectl rollout history deploy/orders -n shop

Medium Task 4: Undo

Why: Rollback.

What to do: Undo to revision 1 and confirm the image.

How:
  1. --to-revision.
Solution
kubectl rollout undo deploy/orders -n shop --to-revision=1

Medium Task 5: Recreate

Why: Strategy.

What to do: Apply billing-deploy.yaml and change VERSION.

How:
  1. set env.
Solution
kubectl set env deploy/billing -n shop VERSION=v3

Medium Task 6: Pause

Why: Batch changes.

What to do: Pause, make two changes, resume, and count revisions.

How:
  1. rollout pause/resume.
Solution
kubectl rollout pause deploy/orders -n shop

Medium Task 7: Restart

Why: Config reload.

What to do: Run rollout restart and find the restartedAt annotation.

How:
  1. jsonpath on template annotations.
Solution
kubectl get deploy orders -n shop -o jsonpath='{.spec.template.metadata.annotations}'

Medium Task 8: Bad image

Why: Safety.

What to do: Apply orders-bad.yaml and prove old pods still serve.

How:
  1. Count Running pods.
Solution
kubectl get pods -n shop -l app=orders

Hard Task 9: Surge math

Why: Understand numbers.

What to do: With replicas 10 and 25%/25%, predict max and min pods, then watch.

How:
  1. Surge rounds up, unavailable rounds down.
Solution
kubectl get pods -n shop -l app=orders -w

Hard Task 10: CI gate

Why: Pipelines.

What to do: Write a script that runs rollout status and undoes on failure.

How:
  1. Check the exit code.
Solution
kubectl rollout status deploy/orders -n shop --timeout=90s || kubectl rollout undo deploy/orders -n shop

Homework: try at home

Roll nginx:1.26 to 1.27 on 6 replicas three times, with surge/unavailable set to 1/0, 0/1 and 50%/50%. Time each with time kubectl rollout status and write which is safest and which is fastest.

Recap

  • A Deployment manages ReplicaSets; each template change is a new revision.
  • maxSurge adds extra pods, maxUnavailable allows missing ones; 1 and 0 means zero downtime.
  • Recreate stops everything first, for apps that cannot run two versions.
  • minReadySeconds and readiness probes make rollouts safe.
  • progressDeadlineSeconds and rollout status catch stuck releases.
  • Undo scales an old ReplicaSet back up. 64.x covers release patterns in depth.

FAQ

Q. Why do old ReplicaSets stay at 0?

They hold old templates so undo is instant. revisionHistoryLimit controls how many.

Q. Does undo create a new revision?

Yes. The old revision number moves to the top as the newest.

Q. Does a ConfigMap change trigger a rollout?

No. Use kubectl rollout restart or put a hash of the config in a template annotation.

Q. Percent or number for maxSurge?

Percent scales with replicas; numbers are clearer for small apps.

Q. Is --record still needed?

No, it is deprecated. Use the change-cause annotation.

Q. What happens if I edit the RS a Deployment owns?

The Deployment overwrites your change. Always edit the Deployment.

Interview questions

  1. How does a rolling update work internally?

    Model answer: The Deployment controller creates a new ReplicaSet and scales it up while scaling the old one down, within maxSurge and maxUnavailable limits, until the new RS has all replicas.

  2. How do you achieve zero downtime?

    Model answer: maxUnavailable 0, maxSurge at least 1, a real readiness probe, minReadySeconds, and graceful shutdown with preStop.

  3. RollingUpdate vs Recreate?

    Model answer: RollingUpdate replaces pods gradually with both versions live; Recreate kills all old pods first, causing brief downtime but never mixing versions.

  4. How do you roll back?

    Model answer: kubectl rollout undo deploy/NAME, or --to-revision=N after checking kubectl rollout history.

  5. How does CI know a deployment failed?

    Model answer: kubectl rollout status returns non-zero when progressDeadlineSeconds is exceeded.

Quick quiz

Q1. Default maxSurge and maxUnavailable?

25% each.

Q2. Which strategy stops all old pods first?

Recreate.

Q3. Does scaling create a revision?

No, only pod template changes.

Q4. Which command rolls pods with no YAML change?

kubectl rollout restart.

Q5. What reason shows a stuck rollout?

ProgressDeadlineExceeded.

Samajla ka?