Chapter 61: Kubernetes Storage in Depth: Volumes, PV, PVC, StorageClass, CSI and Snapshots

61.3 hostPath: using the node's own folders, the types, and why it is risky

By Ravindra Bagale · Last updated: 25 October 2026

Why: the problem

Suppose Ola runs a log agent on every Kubernetes node. The agent must read the container logs that kubelet writes under /var/log/pods on each node. No emptyDir or PVC can give it that, because those files belong to the node, not to any pod. This is what hostPath is for: it mounts a file or folder from the node's own filesystem into a pod. hostPath is powerful and dangerous at the same time. A pod that mounts / or /var/run/containerd/containerd.sock from the host can read every secret on the node or start privileged containers. Data written to a hostPath stays on one node, so if the pod is rescheduled to another node it sees an empty folder. That is why Kubernetes docs say to avoid hostPath for apps and why Pod Security Standards block it at the baseline level. So the rule is simple. Use hostPath only for node-level system agents, like log shippers (Fluent Bit), monitoring (node-exporter) and CSI drivers, almost always in a DaemonSet. Mount it read-only when you can, and use the type field so the pod fails fast if the path is wrong. In this lesson we mount node logs read-only in a DaemonSet, show how data stays stuck on one node, use the type field to catch mistakes, and then block hostPath in a namespace with Pod Security Admission.

What: every term in plain English

• hostPath — A volume that mounts a path from the node's filesystem into the pod.
• type: Directory — The path must already exist as a folder, or the pod fails to start.
• type: DirectoryOrCreate — Creates the folder (mode 0755) if it is missing.
• type: File / FileOrCreate — Same checks for a single file.
• type: Socket — The path must be a UNIX socket, like containerd.sock.
• type: "" (empty) — No check at all. The default, and the most error-prone.
• readOnly — The mount cannot be written from the pod.
• Node affinity of data — hostPath data exists only on the node where it was written.
• Pod Security Admission — Built-in admission that enforces privileged, baseline or restricted levels per namespace.
• local PersistentVolume — A safer, scheduler-aware way to use a node's disk; covered in 61.4.

How: step by step

1. Use the ch61 cluster and namespace storage. Two workers make the node problem visible.
2. Run a DaemonSet that mounts /var/log/pods read-only. One pod per node reads that node's logs.
3. Write to a hostPath from a pod on worker 1. Then force the pod onto worker 2.
4. Try a wrong path with type: Directory. The pod stays in ContainerCreating with a clear event.
5. Label the namespace with Pod Security baseline. New hostPath pods are rejected.
6. Clean up. Remove the labels and pods.

Example 1: Read node logs with a read-only hostPath in a DaemonSet

Goal. Read node logs with a read-only hostPath in a DaemonSet.

log-reader.yaml

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: log-reader
  namespace: storage
spec:
  selector:
    matchLabels:
      app: log-reader
  template:
    metadata:
      labels:
        app: log-reader
    spec:
      containers:
        - name: reader
          image: busybox:1.36
          command: ["sh", "-c", "sleep 3600"]
          volumeMounts:
            - name: podlogs
              mountPath: /host/var/log/pods
              readOnly: true
          resources:
            requests: {cpu: 10m, memory: 16Mi}
            limits: {memory: 32Mi}
      volumes:
        - name: podlogs
          hostPath:
            path: /var/log/pods
            type: Directory

What each line does

• DaemonSet — One reader pod per node, as in 60.5.
• hostPath.path — The node folder where kubelet keeps container logs.
• type: Directory — Fails fast if the folder does not exist.
• readOnly: true — The agent can read logs but never change or delete them.
• /host prefix — A common habit so it is clear the files come from the node.

Green output

$ kubectl apply -f log-reader.yaml
daemonset.apps/log-reader created
$ kubectl get pods -n storage -l app=log-reader -o wide
NAME               READY   STATUS    NODE
log-reader-4kq9z   1/1     Running   ch61-worker
log-reader-x7m2d   1/1     Running   ch61-worker2
$ kubectl exec -n storage log-reader-4kq9z -- ls /host/var/log/pods | head -3
kube-system_kindnet-8vj4r_6a1f...
kube-system_kube-proxy-2xq7s_9d3c...
storage_log-reader-4kq9z_1e7b...
$ kubectl exec -n storage log-reader-4kq9z -- touch /host/var/log/pods/x
touch: /host/var/log/pods/x: Read-only file system

Expected result. Each pod sees only its own node's logs, and the read-only mount protects them. This is how Fluent Bit and Datadog agents collect logs.

Example 2: See hostPath data stay stuck on one node

Goal. See hostPath data stay stuck on one node.

sticky-data.yaml

apiVersion: v1
kind: Pod
metadata:
  name: sticky
  namespace: storage
spec:
  nodeName: ch61-worker
  volumes:
    - name: data
      hostPath:
        path: /srv/ola-demo
        type: DirectoryOrCreate
  containers:
    - name: app
      image: busybox:1.36
      command: ["sh", "-c", "sleep 3600"]
      volumeMounts:
        - name: data
          mountPath: /data
      resources:
        requests: {cpu: 10m, memory: 16Mi}
        limits: {memory: 32Mi}

What each line does

• nodeName — Pins the pod to worker 1 for the first run.
• DirectoryOrCreate — Creates /srv/ola-demo on that node if missing.
• Second run — Change nodeName to ch61-worker2 and apply again.
• Result — The new node has its own empty /srv/ola-demo.
• Lesson — hostPath is not shared storage; data does not follow the pod.

Green output

$ kubectl apply -f sticky-data.yaml
$ kubectl exec -n storage sticky -- sh -c 'echo ride-5521 > /data/rides.txt; cat /data/rides.txt'
ride-5521
$ kubectl delete pod -n storage sticky
$ sed -i 's/nodeName: ch61-worker$/nodeName: ch61-worker2/' sticky-data.yaml
$ kubectl apply -f sticky-data.yaml
$ kubectl exec -n storage sticky -- ls /data
$ docker exec ch61-worker cat /srv/ola-demo/rides.txt
ride-5521

Expected result. On worker 2 the folder was empty. The file still sits on worker 1's disk. A database on hostPath would look like it lost all data after a reschedule.

Example 3: Catch wrong paths with type, then block hostPath with Pod Security

Goal. Catch wrong paths with type, then block hostPath with Pod Security.

bad-path.yaml

apiVersion: v1
kind: Pod
metadata:
  name: bad-path
  namespace: storage
spec:
  volumes:
    - name: sock
      hostPath:
        path: /run/containerd/containerd.sockk
        type: Socket
  containers:
    - name: app
      image: busybox:1.36
      command: ["sh", "-c", "sleep 3600"]
      volumeMounts:
        - name: sock
          mountPath: /run/containerd.sock
      resources:
        requests: {cpu: 10m, memory: 16Mi}
        limits: {memory: 32Mi}

What each line does

• Typo sockk — The path does not exist on the node.
• type: Socket — kubelet checks the type before starting, so the pod never runs with a wrong mount.
• Why sockets are dangerous — Access to containerd.sock means control of every container on the node.
• pod-security.kubernetes.io/enforce=baseline — Rejects any new pod that uses hostPath.
• Exempt namespaces — Keep system agents in a separate namespace like logging with privileged level.

Green output

$ kubectl apply -f bad-path.yaml
$ kubectl describe pod -n storage bad-path | tail -2
  Warning  FailedMount  5s  kubelet  MountVolume.SetUp failed for volume "sock" : hostPath type check failed: /run/containerd/containerd.sockk is not a socket file
$ kubectl delete pod -n storage bad-path
$ kubectl label namespace storage pod-security.kubernetes.io/enforce=baseline
namespace/storage labeled
$ kubectl apply -f sticky-data.yaml
Error from server (Forbidden): error when creating "sticky-data.yaml": pods "sticky" is forbidden: violates PodSecurity "baseline:latest": hostPath volumes (volume "data")

Expected result. The type check turned a silent mistake into a clear event, and the baseline label stopped any app team from using hostPath in this namespace.

Every command, step by step (kind lab with kubectl)

1. Check the nodes

   kubectl get nodes
   

   What it does: Two workers make the node problem visible.

   ch61-control-plane   Ready   control-plane
   ch61-worker          Ready   <none>
   ch61-worker2         Ready   <none>
   

2. Run the log reader

   kubectl apply -f log-reader.yaml
   kubectl rollout status ds/log-reader -n storage
   

   What it does: One pod per worker.

   daemon set "log-reader" successfully rolled out
   

3. Read a log through hostPath

   P=$(kubectl get pod -n storage -l app=log-reader -o name | head -1)
   kubectl exec -n storage $P -- sh -c 'tail -2 /host/var/log/pods/kube-system_kube-proxy*/kube-proxy/0.log'
   

   What it does: Raw kubelet log lines, with timestamps and stream names.

   2026-10-25T01:12:44.210Z stderr F I1025 ... "Starting service config controller"
   2026-10-25T01:12:44.311Z stderr F I1025 ... "Caches are synced"
   

4. Prove the mount is read-only

   kubectl exec -n storage $P -- touch /host/var/log/pods/test
   

   What it does: Writes fail.

   touch: /host/var/log/pods/test: Read-only file system
   

5. Write data on worker 1

   kubectl apply -f sticky-data.yaml
   kubectl exec -n storage sticky -- sh -c 'echo ride-5521 > /data/rides.txt'
   

   What it does: Saved on the node.

6. Look at it from the node

   docker exec ch61-worker ls -l /srv/ola-demo
   

   What it does: kind nodes are Docker containers, so docker exec shows the node disk.

   -rw-r--r--    1 root     root    10 Oct 25 01:20 rides.txt
   

7. Move the pod to worker 2

   kubectl delete pod -n storage sticky
   sed -i 's/nodeName: ch61-worker$/nodeName: ch61-worker2/' sticky-data.yaml
   kubectl apply -f sticky-data.yaml
   kubectl exec -n storage sticky -- ls /data
   

   What it does: Empty folder.

8. Try the wrong socket path

   kubectl apply -f bad-path.yaml
   kubectl get events -n storage --field-selector involvedObject.name=bad-path
   

   What it does: FailedMount with a clear reason.

   Warning   FailedMount   pod/bad-path   hostPath type check failed: ... is not a socket file
   

9. Dry-run the Pod Security level

   kubectl label --dry-run=server --overwrite namespace storage pod-security.kubernetes.io/enforce=baseline
   

   What it does: Lists pods that would break before you enforce.

   Warning: existing pods in namespace "storage" violate the new PodSecurity enforce level "baseline:latest"
   Warning: log-reader-4kq9z (and 2 other pods): hostPath volumes
   

10. Enforce baseline

    kubectl delete ds log-reader -n storage; kubectl delete pod -n storage sticky bad-path --ignore-not-found
    kubectl label namespace storage pod-security.kubernetes.io/enforce=baseline
    kubectl apply -f sticky-data.yaml
    

    What it does: Rejected at admission.

    Error from server (Forbidden): ... violates PodSecurity "baseline:latest": hostPath volumes (volume "data")
    

11. Clean up

    kubectl label namespace storage pod-security.kubernetes.io/enforce-
    docker exec ch61-worker rm -rf /srv/ola-demo
    

    What it does: Remove the label so later labs can run. Keep the cluster for 61.4.

    namespace/storage unlabeled
    

When it is red 1

MountVolume.SetUp failed for volume "data" : hostPath type check failed: /srv/data is not a directory

Cause. type: Directory but the folder is missing on that node.

Fix, step by step

1. Create it on every node, or use DirectoryOrCreate.
2. Check which node the pod went to with -o wide.

When it is red 2

pods "x" is forbidden: violates PodSecurity "baseline:latest": hostPath volumes

Cause. The namespace enforces baseline.

Fix, step by step

1. Move node agents to a dedicated privileged namespace.
2. Use a PVC for app data.

When it is red 3

ls: /data: empty after reschedule

Cause. hostPath data lives on the old node.

Fix, step by step

1. Use a PVC backed by network storage (EBS, EFS).

When it is red 4

touch: /host/var/log/pods/x: Read-only file system

Cause. The mount is read-only, as intended.

Fix, step by step

1. Only remove readOnly if the agent truly must write, and never for system paths.

Practice tasks

kubectl get pods -n storage -l app=log-reader -o wide

kubectl exec -n storage $P -- ls /host/var/log/pods

kubectl exec -n storage $P -- touch /host/var/log/pods/t

docker exec ch61-worker cat /srv/ola-demo/rides.txt

kubectl exec -n storage t -- cat /etc/node-hostname

kubectl describe pod -n storage bad-path

kubectl label --dry-run=server --overwrite ns storage pod-security.kubernetes.io/enforce=baseline

kubectl apply -f sticky-data.yaml

kubectl get ns logging --show-labels

kubectl explain pod.spec.volumes.hostPath

Recap

• hostPath mounts a node's file or folder into a pod.
• Its data stays on that node and does not follow the pod.
• Use the type field to fail fast and readOnly to stay safe.
• Pod Security baseline blocks hostPath for app namespaces.
• Next: PersistentVolumes and PersistentVolumeClaims, the right way to keep app data.

FAQ

Q. Is hostPath ever fine for apps?

Only for single-node test clusters like kind or minikube. Never for production app data.

Q. Why does kind's local-path provisioner use node folders?

It is a test provisioner. It also pins the PV to one node with nodeAffinity.

Q. Is a local PersistentVolume better than hostPath?

Yes. The scheduler knows the node, so pods follow the data.

Q. Can I mount a single file?

Yes, with type: File or FileOrCreate.

Q. Do managed clusters like EKS allow hostPath?

Yes by default, which is why you should enforce Pod Security yourself.

Q. What do Fluent Bit and node-exporter mount?

/var/log, /proc and /sys from the host, read-only.

Interview questions

1. What is hostPath and when is it appropriate?

   Model answer: A node path mounted into a pod; for node agents in DaemonSets.

2. What are the risks of hostPath?

   Model answer: Node takeover via sensitive paths, and data stuck on one node.

3. How do you prevent teams from using hostPath?

   Model answer: Pod Security Admission baseline or restricted on their namespaces, or a policy engine like Kyverno.

4. hostPath vs local PV?

   Model answer: A local PV has nodeAffinity so the scheduler places the pod on the right node.

5. What does type: Socket do?

   Model answer: Checks the path is a UNIX socket before the container starts.

Samajla ka?