Podplane Kubernetes PaaS Components

Podplane components is a collection of Helm charts for the components which run atop a bare Podplane Kubernetes cluster to deliver a Platform-as-a-Service (PaaS) experience.

A "component" is a logical platform capability backed by one or more application Helm charts, and (optionally) sibling CRD Helm charts.

The component Helm charts are reconciled in-cluster by Flux CD, and are managed via the charts/platform-components chart.

A bootstrap chart initialises a bare Kubernetes cluster with the minimal resources required for the cluster to self-manage via Flux CD reconciling the charts/platform-components chart. From then on, platform-components is the entrypoint for selecting installed components and passing each component chart its values.

The platform and bootstrap charts are kept intentionally simple:

• bootstrap installs the minimum number of resources to have Flux CD and the platform-components chart self-manage the cluster.
• platform-components selects components, wires dependencies, and passes chart values.

Components

Core (always installed):

• Cilium for cluster networking & policies (Apache 2.0)
• CoreDNS for DNS resolution (Apache 2.0)
• Flux CD source-controller + helm-controller for in-cluster Helm reconciliation (Apache 2.0). kustomize-controller, notification-controller and the image controllers are intentionally disabled to keep pod count minimal. CRDs for all Flux controllers are shipped however, so users may run additional Flux controllers alongside the system installation.
• Gateway API CRDs for ingress controllers using Gateway API.
• platform-components: the Podplane platform chart that manages reserved namespaces and the Flux HelmRelease resources for every other component.
• platform-rbac: default RBAC and admission policy configuration.
• platform-runtime: the gvisor and runc RuntimeClasses and the policies that decide which one each Pod uses.

Addons (optional, installable via podplane install <name>):

• Agent Sandbox for isolated, stateful singleton workloads such as AI agent runtimes (Apache 2.0).
• Envoy Gateway for ingress (Apache-2.0).
• Snapshot Controller for persistent volume snapshotting (Apache 2.0).
• Kubernetes Metrics Server for resource metrics used by autoscaling and kubectl top (Apache 2.0).
• Node Problem Detector for surfacing node kernel, filesystem, and runtime issues (Apache 2.0).
• Kubernetes Cluster Autoscaler for automatic node scaling through Cluster API (Apache 2.0).

Provider-Specific Core (always installed for certain providers):

• AWS EBS CSI Driver for persistent storage on AWS (Apache 2.0).
• Cluster API core controller and CRDs for declarative infrastructure-provider cluster lifecycle management (Apache 2.0).
• Nstance Operator as the Cluster API infrastructure provider for Nstance-managed machines on AWS and Google Cloud (Apache 2.0).

Usage

After cluster creation, bootstrap Cilium, Flux CD and the platform chart against the current kubectl context:

./bootstrap/apply.sh

This creates the core platform namespaces with their Pod Security and runtime labels, then bootstraps seven component layers in dependency order:

1. platform-runtime (in platform-cluster) — RuntimeClasses and runtime policies, checked before any platform Pod starts
2. platform-cilium-crds (in platform-cluster)
3. platform-cilium (in platform-cilium) — runs under runc; nodes go Ready after this
4. platform-coredns (in platform-coredns)
5. platform-fluxcd-crds (in platform-cluster)
6. platform-fluxcd (in platform-fluxcd)
7. platform-components (as a Flux HelmRelease in platform-components)

From this point on, Flux reconciles platform-components and every other component from the charts/platform-components chart's values, fetched from this repo via a bootstrap GitRepository resource. To enable / disable / reconfigure a component, edit charts/platform-components/values.yaml (or the corresponding HelmRelease in-cluster) and let Flux pick it up.

kubectl get helmreleases -A

Common environment variables to configure bootstrap:

• PLATFORM_INSTALL=minimal|platform|agentic|all — which platform-components install set to enable (bootstrap foundation default minimal; the user-facing Podplane seed default is agentic). platform adds Envoy Gateway, Podplane operator, Secrets Store CSI Driver, the OpenBao Secrets Store CSI provider, Zot, and their CRDs atop minimal. agentic adds Agent Sandbox and its CRDs atop platform. all enables every listed addon.
• CLUSTER_ID=<id> — cluster ID to set on the Podplane operator when using PLATFORM_INSTALL=platform, agentic, or all (default default).
• SPIFFE_TRUST_DOMAIN=<domain> — immutable SPIFFE trust domain for workload identities. Local bootstrap targets default this to <cluster-id>.k8s.localhost.
• DOMAIN=<zone> — supplies the cluster domain to bootstrap consumers. Local bootstrap enables Envoy ingress using the Nstance node ingress certificate; secrets-backed production ingress uses the configured external secrets provider.
• REGISTRY_HOSTNAME=<host> — pull component images via the given registry mirror.

Envoy ingress supports secrets-backed certificates (default) and node-local certificates for tunnel deployments.

Local Testing

Use the Podplane CLI local VM workflow to test components locally. Start a bare local VM from the CLI repository, then bootstrap this checkout into that VM:

cd ../podplane
go run . local start --components none --follow

cd ../components
make git-sync
DOMAIN=default.localhost make agentic

make git-sync snapshots the current components repo checkout (including unstaged changes) into ~/.podplane/cache/deps/git/components.git on the local-dev branch. The Podplane local server exposes that cache to the VM over HTTPS. make agentic reads podplane local status --json to discover the VM Git URL, Flux secretRef, and CA certificate, then runs bootstrap/apply.sh with the agentic component set.

The local bootstrap targets are:

• make minimal for development and testing with only core components.
• make platform for platform components and generating Netsy snapshots for the seeds repo.
• make agentic for the platform baseline plus Agent Sandbox and its CRDs.
• make all for testing all platform, agentic, and additional addons.

After bootstrap, run make git-sync again for manual updates, or make git-watch to sync automatically on file changes:

make git-watch

See the Podplane CLI docs/development.md guide for the full cross-repository local development flow, including dependency caching and seed snapshot creation.

Repository Layout

• charts/<name>/ — component Helm charts. App charts are thin wrappers around upstream charts declared in Chart.yaml dependencies:.
• charts/<name>-crds/ — sibling CRD chart per Helm Best Practices for CRDs. Each ships vendored upstream CRD YAML in templates/external/; updates are handled by the shared scripts/crds Go command.
• charts/platform-components/ — the platform chart that emits Flux HelmRelease resources for components and manages platform namespaces.
• charts/platform-rbac/ — Podplane platform RBAC and admission policies.
• bootstrap/ — render-only bootstrap chart plus apply.sh; brings the cluster from bare to Flux-managed. Creates the GitRepository resource used by all Flux CD HelmRelease resources created.

CRD Compatibility Policy

Podplane-managed CRDs are upgraded only across upstream-supported compatibility windows. Podplane may remove deprecated served CRD versions only in a documented breaking components release, with migration notes.

The shared scripts/crds updater must preserve every previously served CRD API version unless the change is intentionally handled as a breaking release. It downloads or renders upstream CRDs into a temporary directory, compares them against the currently vendored CRDs, and only replaces templates/external/ after the compatibility check passes. The check fails if a CRD is removed or stops serving a previously served version.

Use make update-crds to update every Podplane-managed CRD chart, or make update-crds CHART=<name> for one chart.

Use make check-crd-freshness CHART=nstance-operator-crds to compare vendored CRDs with upstream without changing the checkout.

Validation and Releases

Run make precommit for offline checks, including shared-image version alignment (such as the operator and Envoy Gateway SDS sidecar) and manifest freshness.

Run make ci for chart linting, schema validation, CRD compatibility, image inventory checks, and Go tests. Requires helm, jq, Go, and kubeconform; these checks do not install anything into a cluster. Schema validation skips CRD definitions, but validates custom resources against the schemas shipped in those CRDs.

After changing image versions, run make update-manifests, or make update-manifests COMPONENT=envoy-gateway to update only one component. The script also accepts go run ./scripts/manifests --component envoy-gateway. A single-component update requires an existing manifest and preserves its version, source, and unrelated images. It refreshes shared images while retaining their other component memberships. --check always checks the full inventory and cannot be combined with --component.

Prepare chart packages, the release manifest, and SHA512 checksums locally with:

VERSION=X.Y.Z make release-assets

The output directory (dist/release by default) must be empty. On a v* tag, CI signs the checksums and publishes a new GitHub Release without overwriting an existing release.

Cluster Design Assumptions

The platform defaults assume Podplane's standard dual-stack network design:

• We assume Kubernetes is configured with dual-stack IPv4 + IPv6.

  • Pod IPv4 CIDR block is 100.64.0.0/10, supporting up to 4,194,304 IPv4 addresses. RFC 6598 reserves this CIDR block for Carrier-Grade NAT.

  • Pod IPv6 CIDR block is fd64::/48.

  • Service IPv4 CIDR block is 198.18.0.0/15, supporting up to 131,072 IPv4 addresses.

  • Service IPv6 CIDR block is fdc6::/108.

    • Note that kube-apiserver requires a prefix length >= 108.

  • Both IPv4 CIDR blocks are defined as private networks https://en.wikipedia.org/wiki/Reserved_IP_addresses

  • Both IPv4 CIDR blocks fall within the default set of eBPF-based nonMasqueradeCIDRs https://docs.cilium.io/en/stable/network/concepts/masquerading/

  • Both IPv4 CIDR blocks are configured on kube-controller-manager. The service CIDR blocks are configured on kube-apiserver. We also configure per-Node CIDR blocks with /24 prefix length for IPv4, and /64 prefix length for IPv6.

• We configure Cilium CNI to use Kubernetes IPAM mode.

• CoreDNS runs as a DaemonSet

  • It uses the last service IPv4, 198.19.255.254

  • It uses the last service IPv6, fdc6::ffff

  • The kubelet is configured to use the above two addresses as clusterDNS.

Container Runtimes

Pods run in gVisor by default. platform-runtime enforces this for every new Pod:

• Pods that don't set runtimeClassName get gvisor.
• runc is allowed only in namespaces labelled runtime.podplane.dev/runc: allowed, which only cluster admins can set.
• Any other RuntimeClass, including "", is rejected.

Platform components that need host access gVisor can't provide (host networking, host sockets, mount propagation, or devices) run under runc: Cilium, the CSI drivers and Secrets Store providers, and Node Problem Detector. Their namespaces are listed in runcDefaultNamespaces in charts/platform-runtime/values.yaml, so their Pods default to runc, and set runcAllowed: true in charts/platform-components/values.yaml. A test keeps both lists in sync. Everything else, including CoreDNS and Envoy Gateway, runs in gVisor. Static Pod mirrors in kube-system are exempt, as kubelet runs static Pods with runc.

To see which namespaces allow runc and which runtime each Pod uses:

kubectl get namespaces -l runtime.podplane.dev/runc=allowed
kubectl get pods -A -o custom-columns=NAMESPACE:.metadata.namespace,NAME:.metadata.name,RUNTIME:.spec.runtimeClassName

Learn More

For more, see the official Podplane site at podplane.dev, and the components documentation at podplane.dev/docs/components.

License

Podplane is licensed under the Apache License, Version 2.0. Copyright The Podplane Authors.

See the LICENSE file for details.