Repository navigation
feat(kubernetes): add operator allowlists for sandbox pod annotations and labels - #4497
Conversation
Add allowed_pod_annotations to the Kubernetes driver config. Sandbox template annotations must match an exact key or a <domain>/* entry, and the default empty list rejects all of them. Keys in the openshell.ai and agents.x-k8s.io domains are always rejected. Expose the setting through the server.allowedPodAnnotations Helm value. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
…on-allowlist/matthewgrossman Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> # Conflicts: # deploy/helm/openshell/templates/gateway-config.yaml
|
🌿 Preview your docs: https://nvidia-preview-pr-4497.docs.buildwithfern.com/openshell |
|
Label |
PR Review StatusThe independent review found no blocking issues in the Kubernetes pod annotation allowlist. The operator configuration, validation on both sandbox-create paths, reserved-domain rejection, and Fern docs cover the stated behavior. Blocking findings: None. Carried findings: None. Non-blocking suggestions: None. Branch Checks, Helm Lint, and Trivy Changes passed. I applied Gator metadata
|
johntmyers
added
gator:merge-ready
and removed
gator:watch-pipeline
Assert the exact rejected annotation keys, exercise create_sandbox_inner with a disallowed annotation, and point the rejection error at the Helm gatewayConfig path. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
The rejection error reaches API callers, not operators, so name only the gateway config key. Spell the Helm path in the runtimes docs with the quoted table key to avoid an invalid nested gatewayConfig rendering. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
PR Review StatusThe independent follow-up review found no blocking issues in the new annotation-rejection test or the corrected Helm configuration key. The earlier clean review still applies to the unchanged implementation, and no review obligations remain. Required checks are pending on the current head. Blocking findings: None. Carried findings: None. Gator metadata
|
johntmyers
added
gator:watch-pipeline
Fold the allowlist matching and validation cases into table-driven tests, reuse the json_struct helper for templates, and document why the openshell.ai and agents.x-k8s.io domains are reserved. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
PR Review StatusThe independent follow-up review found no blocking issues in the annotation allowlist test consolidation or reserved-domain documentation clarification. The earlier clean review still applies to the unchanged implementation, and no review obligations remain. Branch Checks, Helm Lint, and E2E are running on the current head. Blocking findings: None. Carried findings: None. Gator metadata
|
johntmyers
added
gator:merge-ready
and removed
gator:watch-pipeline
Sandbox template labels were copied onto workload pods unfiltered. Labels select pods for NetworkPolicy rules and Services, so a caller could place a sandbox under rules meant for other workloads or set OpenShell-owned labels. Add allowed_pod_labels, sharing the allowed_pod_annotations entry format, validation, and reserved openshell.ai / agents.x-k8s.io domains. Empty by default, which rejects every template label. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
matthewgrossman
changed the title
PR Review StatusThe independent Critical-only follow-up review found no new Critical defects in the pod label allowlist or shared annotation/label validation. The earlier reviews continue to cover the unchanged implementation, and no carried findings or concrete maintainer scope decision remain. Branch Checks, Helm Lint, Trivy Changes, and DCO have passed on the current head. Maintainer approval is present, and the current-head E2E workflow is running, including Kubernetes and workspace suites. Gator will monitor the remaining results before returning this PR to merge-ready. Blocking findings: None. Carried findings: None. Gator metadata
|
johntmyers
added
gator:watch-pipeline
johntmyers
added
gator:merge-ready
and removed
gator:watch-pipeline
Monitoring CompleteMonitoring is complete because this PR has merged. Final status: Gator reached merge-ready with no blocking review findings. The final-head Branch Checks, Helm Lint, and E2E gates passed. I am removing the active Gator metadata
|
3 participants
Summary
Add operator-controlled allowlists for the pod annotations and labels that sandbox templates can set on Kubernetes sandbox pods. By default, no template annotations or labels are allowed.
Related Issue
Related: #2102. This PR adds the operator allowlist that applying
--labelvalues to pods would need.Changes
allowed_pod_annotationsandallowed_pod_labelsfields in[openshell.drivers.kubernetes]. Each entry is an exact key (example.com/team,app) or a<domain>/*wildcard, which matches keys in that domain only, not subdomains. Both lists use the same format and validation.InvalidArgument, naming each rejected key). The check runs in bothvalidate_sandbox_createandcreate_sandbox_inner.openshell.aiandagents.x-k8s.io(and their subdomains) are always rejected. The gateway refuses to start if either list includes one of them or a malformed entry.gatewayConfig["openshell.drivers.kubernetes"].allowed_pod_annotations/allowed_pod_labels(no legacyserver.*alias). Standalone driver:--allowed-pod-annotations/OPENSHELL_K8S_ALLOWED_POD_ANNOTATIONSand--allowed-pod-labels/OPENSHELL_K8S_ALLOWED_POD_LABELS.Behavior change: On Kubernetes deployments, sandbox templates that set pod annotations or labels are rejected until an operator allowlists those keys. The
openshellCLI doesn't set template labels or annotations, and reusable workload-template labels are metadata only, so this affects only direct API or SDK callers that setSandboxTemplate.labelsorSandboxTemplate.annotations. It applies to new sandbox creation and to recovery of existing sandboxes: when the gateway recreates a sandbox after a timed-out provisioning attempt, the stored template is validated again. A normal stop and start does not re-validate. Sandbox metadata labels (--label,--selector) and other drivers are unchanged.Testing
cargo test -p openshell-driver-kubernetes(288 passed)cargo clippy -p openshell-driver-kubernetes -p openshell-gateway --all-targetscleanhelm unittest deploy/helm/openshell(330 passed),mise run helm:docs:checkChecklist