Pod Security

Even with perfect access rules, a vulnerable app can be attacked. If an attacker breaks into a container running as root, with special powers, they can try to escape to the node. So the last line of defence is the pod itself. In this lesson, you will lock down containers, and enforce it for a whole namespace.

• Assume an app can be hacked
• Limit what a container can do
• Enforce rules for a namespace

Hardening checklist

There are a few settings that remove most of the danger. Run as a non root user, so a break in does not give root powers. Make the root file system read only, so an attacker cannot drop new programs. Disable privilege escalation. And drop all Linux capabilities, adding back only what is truly needed. Also avoid privileged containers, and host paths. Together these settings make a hacked container much less useful.

A hardened pod

Here is a pod with these settings. At the pod level, run as non root is true, and the user is one thousand. At the container level, allow privilege escalation is false. The root file system is read only. And under capabilities, we drop all. This pod can do its job, and very little else.

apiVersion: v1
kind: Pod
metadata:
  name: hardened
spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 1000
  containers:
  - name: app
    image: busybox:1.36
    command: ["sleep", "3600"]
    securityContext:
      allowPrivilegeEscalation: false
      readOnlyRootFilesystem: true
      capabilities:
        drop: ["ALL"]

Proving the lock

Let's prove it. We apply the pod, and ask which user it runs as. User one thousand, not root. Now we try to create a file in the root folder. The system answers, read only file system. The container is locked down. Next, we enforce rules for a whole namespace, using labels. We create a namespace, and label it as restricted. Now, running a plain engine X pod there is refused, with a list of the violated rules. Nobody can deploy something unsafe.

kubectl apply -f hardened.yaml
pod/hardened created
kubectl exec hardened -- id
uid=1000 gid=0(root) groups=0(root)
kubectl exec hardened -- touch /x
touch: /x: Read-only file system
command terminated with exit code 1
kubectl create namespace locked
namespace/locked created
kubectl label namespace locked pod-security.kubernetes.io/enforce=restricted
namespace/locked labeled
kubectl run bad --image=nginx:1.27 -n locked 2>&1 | cut -c1-100
Error from server (Forbidden): pods "bad" is forbidden: violates PodSecurity "restricted:latest": al

apiVersion: v1
kind: Pod
metadata:
  name: hardened
spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 1000
  containers:
  - name: app
    image: busybox:1.36
    command: ["sleep", "3600"]
    securityContext:
      allowPrivilegeEscalation: false
      readOnlyRootFilesystem: true
      capabilities:
        drop: ["ALL"]

Pod Security Standards

Kubernetes defines three standard levels. Privileged is unrestricted, and only for system components. Baseline blocks the known dangerous settings, and suits most ordinary apps. Restricted follows all the hardening best practices, and is the goal for security sensitive workloads. You apply a level to a namespace with labels, in modes. Enforce rejects violations, warn only prints a warning, and audit logs them. Start with warn, and move to enforce, when apps are ready.

Summary