Skip to content

test(configurator): add repeatable isolated recovery lab - #141

Merged
lweberru merged 1 commit into
mainfrom
test/recovery-lab
Oct 23, 2026
Merged

lweberru merged 1 commit into
mainfrom
test/recovery-lab

Conversation

@lweberru

Copy link
Copy Markdown
Collaborator

Problem and resulting behavior

Recovery acceptance needs a repeatable local environment and deterministic process barriers instead of manual timing. Add a digest-pinned PostgreSQL Compose lab with separate source/restored volumes, internal-only networking, private local backups and no cloud bindings. The controller preserves migration checksums, owners, effective ACLs and RLS, and compares restored application data/functions.

A separate real OpenTofu 1.12.6 fixture interrupts a running Plan at a loopback State read and a saved-plan Apply after an observed checkpoint. The Apply test establishes existing State first, then requires a newly created resource to survive unchanged through fresh Plan/Apply. Controllers terminate only their own synthetic process groups; no production execution bypass is added.

The initially empty local-State variant exposed an incomplete intermediate checkpoint and remains unqualified in #140. Its optional diagnostic fails on missing completed resources; the default passing existing-State test does not claim to solve it. Hosted Runner loss, real S3 locking, cloud reconciliation, full-service restore and real-backup import remain outside this fixture's evidence (#90/#95).

Validation

  • Six isolation/security guard tests passed.
  • Actual local Compose selftest passed repeatedly, including all 66 current SQL migrations and matching contents of 51 tables, relation owners/effective ACLs/RLS and function definitions/owners.
  • Real pinned native Plan interruption and existing-State Apply interruption passed, including fresh Plan/Apply and unchanged retained resource identity.
  • First-Apply diagnostic findings retained in Investigate incomplete first-Apply intermediate checkpoint with OpenTofu 1.12.6 local backend #140; no assertions removed to claim success.
  • EN/DE commands and documentation links checked; git diff --check passed.

Adds the credential-free Isolated recovery lab CI job with cleanup and bilingual operator/workflow documentation. No customer or production operations, no cloud provisioning, no Assistant knowledge changes, no public backup/log artifacts. The local lab is infrastructure as code, not another hosted Dev installation.

@lweberru
lweberru merged commit f4688f0 into main Oct 23, 2026
14 checks passed
@lweberru
lweberru deleted the test/recovery-lab branch October 23, 2026 08:55
lweberru added a commit that referenced this pull request Oct 23, 2026
#143)

## Problem and resulting behavior

The initially empty local-filesystem checkpoint gap in #140 must not be
inferred for remote backends. Extend #141's deterministic local lab with
the real pinned OpenTofu HTTP and S3 backend implementations against
in-memory loopback protocol fixtures.

Each case starts without workload resources, captures both completed
resources in an intermediate checkpoint, hard-stops only its own Apply
process group and verifies a fresh Plan is blocked by the orphaned
remote lock. Only after confirming that child exited does the controller
remove the exact in-memory fixture lock. Fresh Plan/Apply must retain
the newly created resource without replacement or ID change. No
operational force-unlock path is introduced.

## Validation

- Real OpenTofu 1.12.6 HTTP and S3 protocol fixtures passed, including
retained checkpoints, lock-blocked Plan and successful recovery with
unchanged resource identity.
- Six lab isolation/security unit tests passed; Python compile,
bilingual command parity and `git diff --check` passed.
- Credential-free CI now runs both protocol cases with no raw-State/log
artifact publication.

This is protocol-fixture evidence, not STACKIT
authentication/eventual-consistency/live-lock acceptance or hosted
Runner reconciliation. The S3 fixture uses dummy keys; S3 init may
establish an empty State object. The initially empty local-filesystem
variant remains unqualified in #140. No customer or production
Plan/Apply, no cloud provisioning, no Dev deployment or Assistant
changes. Refs #90, #95, #140.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant