fix(helm): render Restricted-compatible security contexts

[lunarwhite]

Summary

Pod Security admission rejects the certgen hook in a namespace enforcing Restricted, blocking the whole release at helm.sh/hook-weight: "-20". This adds the missing controls to both certgen Jobs and to the gateway container, and makes the hook context overridable.

Related Issue

Closes #3215

Changes

• templates/certgen.yaml: add runAsNonRoot and seccompProfile.type: RuntimeDefault to the hook container, rendered from values rather than hardcoded. Apply the same context to the -backend-ca post-install hook.
• values.yaml: add seccompProfile.type: RuntimeDefault to the gateway securityContext; add pkiInitJob.securityContext, defaulted to what the hook previously hardcoded.
• tests/certgen_test.yaml, tests/gateway_pod_security_context_test.yaml: assert the Restricted controls on all three containers, and that the certgen hook sets no runAsUser.
• docs/kubernetes/setup.mdx: Pod Security section. skills/debug-openshell-cluster/SKILL.md: troubleshooting row for the PodSecurity rejection. README.md: regenerated.

Design notes

The gateway was broken too, and the hook was hiding it. The hook fails first at weight -20, so the gateway's own missing seccompProfile never surfaced. A hook-only fix would leave the release recorded as deployed while the workload never comes up.

Both certgen Jobs are covered. <release>-certgen-backend-ca, the post-install hook that renders when cert-manager and grpcRoute.backendTLSPolicy are both enabled, had the identical gap. Both Jobs now render from one pkiInitJob.securityContext, so the two cannot drift again, which is what caused this bug.

Field placement is load-bearing. allowPrivilegeEscalation and capabilities.drop have no pod-level equivalent, so all four controls live together on the container. The gateway profile must be on securityContext rather than podSecurityContext because ci/values-openshift-scc.yaml nulls the pod block. runAsUser stays unset on the hook: Dockerfile.gateway declares a numeric USER 1000:1000, which is what kubelet needs to admit runAsNonRoot without an explicit UID, and setting one would break OpenShift restricted-v2, which assigns the namespace UID itself. A notExists assertion pins that.

Container-level override only. This satisfies acceptance criterion 3, which is conditional ("if operators need to override"). No pod-level knob is added: the hook pods run one container and mount no volumes, so the pod-scoped fields have nothing to act on. The gateway has podSecurityContext only because its StatefulSet mounts a PVC. Defaults are unchanged, so no existing install is affected.

Scope

This covers the chart's own pods: the gateway and both certgen hooks. Sandbox pods come from the Kubernetes compute driver, not the chart. Under the current sandbox architecture they are capability-free and non-root, but RFC 0012 does not guarantee Pod Security Standards compliance, so the docs make no claim about them.

Known risks

• helm upgrade rolls the gateway and applies the CRI syscall filter to a process that previously ran unconfined on most clusters. The gateway does no mount, namespace, or bpf work, and e2e installs chart defaults on every run.
• pkiInitJob.securityContext: null renders an empty context and silently reverts the fix. This matches how _gateway-workload.tpl already treats .Values.securityContext, so it is consistent with the existing pattern rather than a new failure mode.

Testing

• mise run pre-commit passes
• Unit tests added/updated
• E2E tests added/updated (if applicable)

Tests added (unit only; chart rendering has no integration layer, and no e2e/ files changed): the four Restricted controls on the certgen hook and the gateway container, runAsNonRoot and seccompProfile.type on the backend CA hook, and notExists on the certgen hook's runAsUser.

Each was confirmed to fail without the template change:

FAIL  certgen hook
  - renders a Restricted-compatible securityContext on the certgen container
      unknown path spec.template.spec.containers[0].securityContext.runAsNonRoot
      unknown path spec.template.spec.containers[0].securityContext.seccompProfile.type
  - renders a Restricted-compatible securityContext on the backend CA hook
      unknown path spec.template.spec.containers[0].securityContext.runAsNonRoot
      unknown path spec.template.spec.containers[0].securityContext.seccompProfile.type
FAIL  gateway pod securityContext
  - renders a Restricted-compatible container securityContext by default
      unknown path spec.template.spec.containers[0].securityContext.seccompProfile.type

Admission verification

kind v0.32.0, Kubernetes v1.36.1 (the report is v1.36.4), namespace labeled pod-security.kubernetes.io/enforce=restricted and enforce-version=latest. PodSecurity enforces on Pods rather than on the workloads that create them, so each rendered pod template was submitted as a bare Pod with kubectl apply --dry-run=server.

Before, both pods are rejected, reproducing the report and exposing the masked gateway failure:

pods "pss-before-job" is forbidden: violates PodSecurity "restricted:latest":
  runAsNonRoot != true (pod or container "certgen" must set securityContext.runAsNonRoot=true),
  seccompProfile (pod or container "certgen" must set securityContext.seccompProfile.type
  to "RuntimeDefault" or "Localhost")

pods "pss-before-statefulset" is forbidden: violates PodSecurity "restricted:latest":
  seccompProfile (pod or container "openshell-gateway" must set
  securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")

After, both are admitted and the workloads emit no warnings:

pod/pss-after-job created (server dry run)
pod/pss-after-statefulset created (server dry run)
statefulset.apps/openshell created (server dry run)      # no warning
job.batch/openshell-certgen created (server dry run)      # no warning

The hook then ran for real in the Restricted namespace, completing in 17s and creating all three PKI secrets:

job.batch/openshell-certgen   Complete   1/1   17s
openshell-client-tls   kubernetes.io/tls   3
openshell-jwt-keys     Opaque              3
openshell-server-tls   kubernetes.io/tls   3

INFO openshell_server::certgen: PKI secrets created.
  namespace=openshell server=openshell-server-tls
  client=openshell-client-tls jwt=openshell-jwt-keys

All ci/values-*.yaml overlays, plus the backendTLSPolicy, upstream proxy CA, SPIFFE, and Deployment combinations, were checked against the full Baseline and Restricted rule set, including volume types, host namespaces, and host ports. Every rendered container pass, including values-openshift-scc, where the overlay keeps seccompProfile and drops runAsUser under Helm's deep merge. On main without this change, the same check flags only the hook and gateway gaps fixed here.

Seccomp on the long-running gateway

RuntimeDefault newly applies the runtime's syscall filter to the gateway, so it was exercised rather than argued. mise run e2e:kubernetes on kind v1.36.1 with ghcr.io/nvidia/openshell/gateway:dev:

openshell-0   1/1   Running
INFO openshell_server::cli: Starting OpenShell server bind=0.0.0.0:8080
INFO openshell_server::compute: Compute driver connected driver=kubernetes
INFO openshell_server: Created default workspace
INFO openshell_server::gateway_listener: Gateway listener bound address=0.0.0.0:8080
INFO openshell_server: Health server listening address=0.0.0.0:8081
INFO openshell_server: Metrics server listening address=0.0.0.0:9090

The gateway reached Ready under RuntimeDefault, bound all three listeners, provisioned its SQLite PVC, and served CreateSandbox, ListSandboxes, GetSandbox, and DeleteSandbox with HTTP 200 across a seven-minute reconcile window.

The suite does not pass end to end on that host: smoke/create times out with DependenciesNotReady: Pod exists with phase: Failed. That is a sandbox-pod failure, pre-existing and unrelated, the same lane at the base commit produces the byte-identical diagnostic. Sandbox pods are built by the Kubernetes compute driver at runtime, and .Values.securityContext has exactly one consumer, _gateway-workload.tpl, so this change cannot reach them.

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)
• Architecture docs updated (if applicable)

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[lunarwhite]

Hi @johntmyers, would you please add this to your review queue? It's part of the 0.1.15 milestone. Thanks.

[lunarwhite]

Hi @TaylorMutch (it's really great to see you're back! I hope everything went well), would you be interested in reviewing this PR?

[github-actions]

Label test:e2e applied, but pull-request/3340 does not exist yet. A maintainer needs to comment /ok to test b50de7d15e009655be060562ca3d2b022b42d958 to mirror this PR. Once the mirror exists, re-apply the label or re-run Branch E2E Checks from the Actions tab.

[TaylorMutch]

/ok to test b50de7d

[TaylorMutch]

@lunarwhite can you rebase on top of main to see if we can get past the E2E flakes?

[TaylorMutch]

/ok to test 0163cdc

[github-actions]

Compatibility review (advisory)

Unaddressed breaking changes

The certgen hook now requires non-root execution by default. Existing releases using a root-based custom gateway image can fail during install or upgrade unless they add a new override.

1. helm — NOT ACKED

The certgen hooks previously accepted the image’s root user but now set pkiInitJob.securityContext.runAsNonRoot=true.

• Impact: Releases using a root-based gateway.image and overriding only securityContext.runAsNonRoot=false can have the pre-install or pre-upgrade certgen Job rejected before the gateway starts.
• Action: Preserve root-image upgrades or add explicit migration guidance requiring pkiInitJob.securityContext.runAsNonRoot=false before upgrading, with rebuilding the image as non-root as the preferred migration.
• PR: fix(helm): render Restricted-compatible security contexts #3340

Review details

• Execution: complete
• Candidate: PR #3340 (4efd56ea7fa8b8c3c89a3b07f80fd19ede81d02c)
• Baseline: 90161fbd7d87b6c07b88183b801b30b4925d84d7 (90161fbd7d87b6c07b88183b801b30b4925d84d7)
• Reviewed: rust-sdk, python-sdk, go-sdk, typescript-sdk, api-behavior, cli, configuration, policy, helm, state

Advisory review; publication gates are unchanged.

Branch check run