From 8b19d3b4a4faa359baad5c04ee7a62d8e394fcdf Mon Sep 17 00:00:00 2001 From: Itay Grudev Date: Thu, 18 Dec 2025 19:41:54 +0200 Subject: [PATCH 1/4] docs(cluster): Kyverno Chainsaw testing documentation Signed-off-by: Itay Grudev --- charts/cluster/TESTING.md | 104 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) create mode 100644 charts/cluster/TESTING.md diff --git a/charts/cluster/TESTING.md b/charts/cluster/TESTING.md new file mode 100644 index 0000000000..ff4b5997a2 --- /dev/null +++ b/charts/cluster/TESTING.md @@ -0,0 +1,104 @@ +# Testing + +This chart uses Kyverno Chainsaw and implements end-to-end (E2E) tests for common features. Not everything is tested because of inadequate tooling — for example, local simulation of Azure and Google Cloud Storage. We do test S3 via MinIO. Our aim is that every critical feature that is technically feasible to test is covered. + +We use a local kind cluster (minikube also works) and provision prerequisites such as the CloudNativePG operator, Prometheus CRDs, and MinIO. Then we run the `chainsaw` utility, which executes the individual tests. It can run tests in parallel, which is essential because some tests take over five minutes to complete. + +## Procedure + +1. Create a kind cluster. + + ```bash + kind delete cluster kind + ``` + +2. Install the CloudNativePG operator + + ```bash + helm dependency update charts/cloudnative-pg + helm upgrade \ + --install \ + --namespace $NAMESPACE \ + --create-namespace \ + --set config.clusterWide=$CLUSTER_WIDE \ + --wait \ + cnpg charts/cloudnative-pg + ``` + +3. Install the Prometheus CRDs (optional, but required for monitoring tests) + + ```bash + helm repo add prometheus-community https://prometheus-community.github.io/helm-charts + helm install prometheus-crds prometheus-community/prometheus-operator-crds + ``` + +4. Install MinIO (optional, but required for backup/recovery tests). + + ```bash + helm repo add minio-operator https://operator.min.io + helm upgrade \ + --install \ + --namespace minio-system \ + --create-namespace \ + --wait \ + operator minio-operator/operator + + helm upgrade \ + --install \ + --namespace minio \ + --create-namespace \ + --wait \ + --values ./.github/minio.yaml \ + tenant minio-operator/tenant + ``` + +5. Install Kyverno Chainsaw + + Refer to the [Kyverno Chainsaw Installation](https://kyverno.io/blog/2023/12/12/kyverno-chainsaw-the-ultimate-end-to-end-testing-tool/#install-chainsaw) documentation for platform specific instructions. + + You can also install Kyverno Chainsaw from source if you have _Go_ installed: + + ```bash + go install github.com/kyverno/chainsaw@latest + ``` + +6. Run the tests + + To run the whole test suite: + + ```bash + chainsaw test charts/cluster + ``` + + To run a specific test, specify its directory path. Example: + + ```bash + chainsaw test charts/cluster/test/postgresql-cluster-configuration + ``` + +## Test structure + +We are only going to outline the test structure here. Refer to the [Kyverno Chainsaw](https://kyverno.github.io/chainsaw/latest/quick-start/) documentation for full reference and capabilities. +The tests are located in the `test` directory. Each test has its own subdirectory. Because clusters take time to provision, where it makes sense tests should be combined and executed sequentially on the same cluster. One exception is critical functionality such as backup/restore, which should be tested independently. + +Inside each test there is a `chainsaw-test.yaml` file that outlines the steps of that particular test. Here are some tips for writing tests for CloudNativePG: + +* Whenever you're adding new features, make sure at the very least to update the `postgresql-cluster-configuration` test that verifies that all non-default configuration options are passed and applied correctly. We're unlikely to merge PRs that don't have their own test or update this one. +* Always manually uninstall Helm chart resources to speed up cleanup. +* Where applicable, and for steps likely to fail, add `catch` statements. For example: + + ```yaml + catch: + - describe: + apiVersion: batch/v1 + kind: Job + - describe: + apiVersion: postgresql.cnpg.io/v1 + kind: Cluster + - podLogs: + selector: batch.kubernetes.io/job-name=your-job-name + ``` + + This will substantially help with debugging later. +* Provide useful step descriptions to aid in understanding test failures. +* Use reasonable test timeouts so tests fail if something isn't finished after 5–10 minutes. Aim for tests to complete within 10 minutes. From e99a1f5312675c1538d373edd199e4102b8e1c06 Mon Sep 17 00:00:00 2001 From: Itay Grudev Date: Fri, 19 Dec 2025 09:54:52 +0000 Subject: [PATCH 2/4] Fixed errata kind create cluster Signed-off-by: Itay Grudev --- charts/cluster/TESTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/charts/cluster/TESTING.md b/charts/cluster/TESTING.md index ff4b5997a2..28102c8151 100644 --- a/charts/cluster/TESTING.md +++ b/charts/cluster/TESTING.md @@ -9,7 +9,7 @@ We use a local kind cluster (minikube also works) and provision prerequisites su 1. Create a kind cluster. ```bash - kind delete cluster kind + kind delete cluster ``` 2. Install the CloudNativePG operator From 9ebb743ae3f3b2c1ae27b1e7b3931b53e5e6dd60 Mon Sep 17 00:00:00 2001 From: Itay Grudev Date: Fri, 19 Dec 2025 09:56:06 +0000 Subject: [PATCH 3/4] Update TESTING.md Signed-off-by: Itay Grudev --- charts/cluster/TESTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/charts/cluster/TESTING.md b/charts/cluster/TESTING.md index 28102c8151..8fd12f4607 100644 --- a/charts/cluster/TESTING.md +++ b/charts/cluster/TESTING.md @@ -9,7 +9,7 @@ We use a local kind cluster (minikube also works) and provision prerequisites su 1. Create a kind cluster. ```bash - kind delete cluster + kind create cluster ``` 2. Install the CloudNativePG operator From eeb260e21a18602d82d239122724c633924666b2 Mon Sep 17 00:00:00 2001 From: Danish Date: Wed, 9 Sep 2026 14:00:46 +0530 Subject: [PATCH 4/4] review: update doc (#1007) Fixing the doc with following. - update with cert-manager and barman-cloud-plugin setup( since whole suit needs this) - fixing broken link( installation of kyverno chainsaw) - fixing env vars Signed-off-by: Danish --- charts/cluster/TESTING.md | 31 ++++++++++++++++++++++++------- 1 file changed, 24 insertions(+), 7 deletions(-) diff --git a/charts/cluster/TESTING.md b/charts/cluster/TESTING.md index 8fd12f4607..632f77be5b 100644 --- a/charts/cluster/TESTING.md +++ b/charts/cluster/TESTING.md @@ -9,7 +9,7 @@ We use a local kind cluster (minikube also works) and provision prerequisites su 1. Create a kind cluster. ```bash - kind create cluster + kind create cluster --name chart-test ``` 2. Install the CloudNativePG operator @@ -18,9 +18,9 @@ We use a local kind cluster (minikube also works) and provision prerequisites su helm dependency update charts/cloudnative-pg helm upgrade \ --install \ - --namespace $NAMESPACE \ + --namespace cnpg-system \ --create-namespace \ - --set config.clusterWide=$CLUSTER_WIDE \ + --set config.clusterWide=true \ --wait \ cnpg charts/cloudnative-pg ``` @@ -32,7 +32,24 @@ We use a local kind cluster (minikube also works) and provision prerequisites su helm install prometheus-crds prometheus-community/prometheus-operator-crds ``` -4. Install MinIO (optional, but required for backup/recovery tests). +4. Install Cert-Manager and Barman plugin (optional, but required for backup/recovery tests). + + ```bash + helm repo add jetstack https://charts.jetstack.io + helm install \ + cert-manager jetstack/cert-manager \ + --namespace cert-manager \ + --create-namespace \ + --version v1.21.1 \ + --set crds.enabled=true + ``` + ```bash + helm upgrade --install plugin-barman-cloud \ + --namespace cnpg-system \ + charts/plugin-barman-cloud + ``` + +5. Install MinIO (optional, but required for backup/recovery tests). ```bash helm repo add minio-operator https://operator.min.io @@ -52,9 +69,9 @@ We use a local kind cluster (minikube also works) and provision prerequisites su tenant minio-operator/tenant ``` -5. Install Kyverno Chainsaw +6. Install Kyverno Chainsaw - Refer to the [Kyverno Chainsaw Installation](https://kyverno.io/blog/2023/12/12/kyverno-chainsaw-the-ultimate-end-to-end-testing-tool/#install-chainsaw) documentation for platform specific instructions. + Refer to the [Kyverno Chainsaw Installation](https://kyverno.github.io/chainsaw/latest/quick-start/install/) documentation for platform specific instructions. You can also install Kyverno Chainsaw from source if you have _Go_ installed: @@ -62,7 +79,7 @@ We use a local kind cluster (minikube also works) and provision prerequisites su go install github.com/kyverno/chainsaw@latest ``` -6. Run the tests +7. Run the tests To run the whole test suite: