Chapter 61: Kubernetes Storage in Depth: Volumes, PV, PVC, StorageClass, CSI and Snapshots

61.1 Volumes overview: why containers lose data and how Kubernetes keeps it

By Ravindra Bagale · Last updated: 25 October 2026

Why: the problem

Suppose Zomato runs a small image-resize service. A user uploads a restaurant photo, the pod saves it to /uploads, and a second container in the same pod makes thumbnails. One night the container hits its memory limit and restarts. Every photo in /uploads is gone. The thumbnail container never saw any of them anyway, because each container has its own filesystem. This is the first storage surprise in Kubernetes: a container's filesystem is temporary. It is built fresh from the image every time the container starts. Anything written there dies with that container. Kubernetes fixes this with volumes. A volume is a directory that Kubernetes attaches to a pod, and containers mount it at a path. Different volume types live for different lengths of time: some for one pod's life (emptyDir), some for the node's life (hostPath), and some forever, until you delete them (PersistentVolumes). In this lesson we prove the problem with a lab, then try the three main families side by side, so the rest of chapter 61 has a clear map.

What: every term in plain English

• Container filesystem — The writable layer on top of the image. Deleted when the container is replaced.
• Volume — A directory attached to a pod, declared in spec.volumes.
• volumeMounts — Where a container sees a volume, set per container.
• emptyDir — Empty folder created with the pod, deleted with the pod.
• hostPath — A folder on the node's own disk.
• PersistentVolume (PV) — A real piece of storage in the cluster, like an EBS disk.
• PersistentVolumeClaim (PVC) — A pod's request for storage: size and access mode.
• StorageClass — A template that creates PVs on demand.
• CSI driver — A plugin that talks to the storage system, like the AWS EBS CSI driver.
• Ephemeral vs persistent — Ephemeral dies with the pod; persistent outlives it.

How: step by step

1. Create a kind cluster for chapter 61. One control plane and two workers.
2. Write a file inside a container and restart it. The file is gone.
3. Add an emptyDir and repeat. The file survives a container restart, not a pod delete.
4. Add a PVC and delete the pod. The file survives even the pod delete.
5. Map each type to a lifetime. Container, pod, node or cluster.
6. Clean up. Delete the namespace; look at what PVs remain.

Example 1: Create the chapter 61 cluster and prove the container filesystem is temporary

Goal. Create the chapter 61 cluster and prove the container filesystem is temporary.

kind-ch61.yaml

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: ch61
nodes:
  - role: control-plane
  - role: worker
  - role: worker

no-volume.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: storage
---
apiVersion: v1
kind: Pod
metadata:
  name: no-volume
  namespace: storage
spec:
  containers:
    - name: app
      image: busybox:1.36
      command: ["sh", "-c", "sleep 3600"]
      resources:
        requests: {cpu: 10m, memory: 16Mi}
        limits: {memory: 32Mi}

What each line does

• name: ch61 — A new cluster, so earlier chapters do not interfere.
• Two workers — Later lessons show data tied to one node.
• Namespace storage — All chapter 61 labs live here.
• No volumes — The app writes only to its own container filesystem.
• kill 1 — Killing PID 1 makes kubelet restart the container in the same pod.

Green output

$ kind create cluster --config kind-ch61.yaml
Creating cluster "ch61" ...
$ kubectl apply -f no-volume.yaml
$ kubectl exec -n storage no-volume -- sh -c 'echo order-1001 > /tmp/orders.txt; cat /tmp/orders.txt'
order-1001
$ kubectl exec -n storage no-volume -- kill 1
$ kubectl get pod -n storage no-volume
NAME        READY   STATUS    RESTARTS
no-volume   1/1     Running   1
$ kubectl exec -n storage no-volume -- cat /tmp/orders.txt
cat: can't open '/tmp/orders.txt': No such file or directory

Expected result. Same pod, restart count 1, file gone. The new container started fresh from the image.

Example 2: Share data between two containers and survive a container restart with emptyDir

Goal. Share data between two containers and survive a container restart with emptyDir.

emptydir-share.yaml

apiVersion: v1
kind: Pod
metadata:
  name: resize
  namespace: storage
spec:
  volumes:
    - name: uploads
      emptyDir: {}
  containers:
    - name: uploader
      image: busybox:1.36
      command: ["sh", "-c", "while true; do date >> /uploads/photos.log; sleep 5; done"]
      volumeMounts:
        - name: uploads
          mountPath: /uploads
      resources:
        requests: {cpu: 10m, memory: 16Mi}
        limits: {memory: 32Mi}
    - name: thumbnailer
      image: busybox:1.36
      command: ["sh", "-c", "sleep 3600"]
      volumeMounts:
        - name: uploads
          mountPath: /in
          readOnly: true
      resources:
        requests: {cpu: 10m, memory: 16Mi}
        limits: {memory: 32Mi}

What each line does

• spec.volumes — Declared once per pod.
• emptyDir: {} — Created empty on the node when the pod starts.
• Two mounts — The same folder at /uploads and /in.
• readOnly: true — The thumbnailer can read but not change uploads.
• Lifetime — Survives container restarts; deleted with the pod.

Green output

$ kubectl apply -f emptydir-share.yaml
$ sleep 15; kubectl exec -n storage resize -c thumbnailer -- wc -l /in/photos.log
3 /in/photos.log
$ kubectl exec -n storage resize -c uploader -- kill 1
$ sleep 10; kubectl exec -n storage resize -c thumbnailer -- wc -l /in/photos.log
5 /in/photos.log
$ kubectl delete pod -n storage resize; kubectl apply -f emptydir-share.yaml
$ sleep 6; kubectl exec -n storage resize -c thumbnailer -- wc -l /in/photos.log
1 /in/photos.log

Expected result. Lines kept growing after the container restart, so the data survived. After the pod was deleted, the count started again from 1.

Example 3: Keep data after the pod is deleted with a PVC

Goal. Keep data after the pod is deleted with a PVC.

pvc-demo.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: orders-data
  namespace: storage
spec:
  accessModes: ["ReadWriteOnce"]
  resources:
    requests:
      storage: 100Mi
---
apiVersion: v1
kind: Pod
metadata:
  name: orders
  namespace: storage
spec:
  volumes:
    - name: data
      persistentVolumeClaim:
        claimName: orders-data
  containers:
    - name: app
      image: busybox:1.36
      command: ["sh", "-c", "sleep 3600"]
      volumeMounts:
        - name: data
          mountPath: /data
      resources:
        requests: {cpu: 10m, memory: 16Mi}
        limits: {memory: 32Mi}

What each line does

• PersistentVolumeClaim — Asks for 100Mi with ReadWriteOnce.
• No storageClassName — Uses the default class; on kind that is standard (local-path).
• claimName — The pod points to the claim, not the disk.
• Dynamic provisioning — The PV is created automatically when the pod is scheduled.
• Lifetime — Data lives until the PVC (and with Delete policy, the PV) is deleted.

Green output

$ kubectl apply -f pvc-demo.yaml
$ kubectl get pvc -n storage
NAME          STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS
orders-data   Bound    pvc-7f3c2a91-5b6e-4d8a-9c1f-2e4b6a8d0c13   100Mi      RWO            standard
$ kubectl exec -n storage orders -- sh -c 'echo order-1001 > /data/orders.txt'
$ kubectl delete pod -n storage orders
$ kubectl apply -f pvc-demo.yaml
$ kubectl exec -n storage orders -- cat /data/orders.txt
order-1001

Expected result. The pod was deleted and recreated, and the order is still there. That is the core of every database on Kubernetes.

Every command, step by step (kind lab with kubectl)

1. Create the cluster

   kind create cluster --config kind-ch61.yaml
   kubectl get nodes
   

   What it does: Three nodes ready.

   ch61-control-plane   Ready   control-plane
   ch61-worker          Ready   <none>
   ch61-worker2         Ready   <none>
   

2. See the default StorageClass

   kubectl get storageclass
   

   What it does: kind ships local-path as default.

   NAME                 PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE
   standard (default)   rancher.io/local-path   Delete          WaitForFirstConsumer
   

3. Prove the loss

   kubectl apply -f no-volume.yaml
   kubectl exec -n storage no-volume -- sh -c 'echo hi > /tmp/a; kill 1'
   sleep 5; kubectl exec -n storage no-volume -- ls /tmp
   

   What it does: Empty after restart.

4. Share with emptyDir

   kubectl apply -f emptydir-share.yaml
   kubectl exec -n storage resize -c thumbnailer -- ls /in
   

   What it does: Both containers see the same file.

   photos.log
   

5. Find emptyDir on the node

   kubectl get pod -n storage resize -o jsonpath='{.metadata.uid}{"\n"}'
   docker exec ch61-worker ls /var/lib/kubelet/pods/<uid>/volumes/kubernetes.io~empty-dir/
   

   What it does: Where the folder really lives (use the node shown by -o wide).

   uploads
   

6. Create the PVC

   kubectl apply -f pvc-demo.yaml
   kubectl get pv,pvc -n storage
   

   What it does: Bound PV and PVC.

   persistentvolumeclaim/orders-data   Bound   100Mi   RWO   standard
   

7. Survive a pod delete

   kubectl exec -n storage orders -- sh -c 'echo order-1001 > /data/orders.txt'
   kubectl delete pod -n storage orders; kubectl apply -f pvc-demo.yaml
   kubectl exec -n storage orders -- cat /data/orders.txt
   

   What it does: Data is still there.

   order-1001
   

8. List volume types in a pod

   kubectl get pod -n storage resize -o jsonpath='{.spec.volumes}'
   

   What it does: Includes the auto-added service account token volume.

   [{"emptyDir":{},"name":"uploads"},{"name":"kube-api-access-x7k2p","projected":{...}}]
   

9. Clean up

   kubectl delete namespace storage
   kubectl get pv
   

   What it does: With Delete policy the PV goes away with its PVC. Keep the cluster for 61.2.

   No resources found
   

When it is red 1

cat: can't open '/tmp/orders.txt': No such file or directory

Cause. The file was in the container filesystem and the container restarted.

Fix, step by step

1. Mount a volume at that path.

When it is red 2

spec.containers[0].volumeMounts[0].name: Not found: "upload"

Cause. The mount name does not match a name in spec.volumes.

Fix, step by step

1. Make the names identical.

When it is red 3

orders-data   Pending   ...   waiting for first consumer to be created before binding

Cause. The StorageClass uses WaitForFirstConsumer; no pod uses the claim yet.

Fix, step by step

1. Create the pod; it binds then. This is normal.

When it is red 4

sh: can't create /in/photos.log: Read-only file system

Cause. The mount is readOnly: true.

Fix, step by step

1. Write through a container that mounts it read-write.

Practice tasks

kind create cluster --config kind-ch61.yaml

kubectl exec -n storage no-volume -- kill 1

kubectl get sc standard -o yaml

kubectl exec -n storage resize -c thumbnailer -- wc -l /in/photos.log

kubectl exec -n storage resize -c thumbnailer -- touch /in/x

docker exec ch61-worker ls /var/lib/kubelet/pods

kubectl exec -n storage orders -- cat /data/orders.txt

kubectl get pv -o yaml

kubectl get pv -o wide

kubectl get pv,pvc -A

Recap

• Container files die with the container.
• Volumes are declared in spec.volumes and mounted with volumeMounts.
• emptyDir lives as long as the pod.
• PVCs ask for storage; PVs are the storage; StorageClasses create PVs.
• Next: emptyDir in depth, including memory-backed volumes and size limits.

FAQ

Q. Is a volume the same as a Docker volume?

Similar idea, but Kubernetes manages it and ties it to a pod.

Q. Can two pods share an emptyDir?

No, only containers in the same pod.

Q. Is local-path safe for production?

No, data stays on one node. Use EBS, Persistent Disk or Azure Disk through CSI.

Q. Do ConfigMaps count as volumes?

Yes, they can be mounted; see 61.6.

Q. What about S3?

S3 is object storage, used through an SDK or the Mountpoint for S3 CSI driver, not as a normal disk.

Q. Why is there a kube-api-access volume?

Kubernetes adds a projected service account token to every pod.

Q. What happens to volumes when a node dies?

emptyDir and hostPath data on that node is lost to the pod. Network disks like EBS detach and attach to the new node, which can take a few minutes.

Q. How do I see which volumes a running pod uses?

kubectl describe pod shows a Volumes section with each name, type and source, and the Mounts line under each container shows the paths.

Q. Which volume types should a beginner learn first?

emptyDir, ConfigMap and Secret volumes, and PVCs. They cover almost every app. hostPath and CSI details come later in this chapter.

Interview questions

1. Why do containers lose data on restart?

   Model answer: Each new container starts from the image with a fresh writable layer.

2. Difference between a volume and a PersistentVolume?

   Model answer: A volume is in the pod spec and may be temporary; a PV is a cluster object with its own life.

3. What is the lifetime of an emptyDir?

   Model answer: The life of the pod on that node.

4. How does a pod get persistent storage?

   Model answer: It references a PVC; the PVC binds to a PV, often created by a StorageClass.

5. Which volume types would you use for a sidecar sharing logs?

   Model answer: emptyDir.

Samajla ka?