Repository navigation
Configurator MVP: operational acceptance and feature-branch merge readiness #95
Description
Source and release checkpoint (2026-10-23)
#135 merged after all 13 CI checks passed. Published recovery migration 064 remains unchanged; new source/identity migrations follow as 065/066. The user-authorized qualified Dev release 39925017270 stopped before build/deployment because the repeated real-model suite failed the same DNS completion case twice (122/124 passed); the continuous journey passed. Current Dev therefore remains on #136. A bounded DNS intent correction is under investigation; no release guard is bypassed and no new cloud qualification is claimed.
#141 is also merged: isolated Compose database restore and deterministic local native interruption CI passed. First-Apply local-State completeness remains separately open in #140; hosted Runner/S3 and complete-service recovery are not established by those local fixtures.
Public Application acceptance (2026-10-23)
The user reports successful creation of a Public Application Landing Zone including an Observability instance. Both the real Plan and Apply succeeded on the first attempt. Together with the Corporate acceptance and same-order partial-Apply recovery already recorded in #93, both Public and Corporate provisioning paths now have positive real-user acceptance. This does not qualify the effective-egress Observability ACL feature in #96, Automatic Corporate IPv4 in #135, cross-instance lock contention, upgrades, quotas or hard worker-loss/restore behavior. No customer Plan/Apply was executed by the agent.
Current acceptance scope decision (2026-10-23)
At the user's explicit decision, a second real STACKIT organization is removed from the current Dev/MVP acceptance scope because another organization is not available. Cross-organization live acceptance is deferred to a future expressly authorized customer validation; it is not recorded as passed and does not block completion of the current scope. Existing tenant/organization authorization and isolation guards remain required and unchanged.
Current live acceptance uses the available organization with separate Platform Owner and Application Owner accounts. Role/owner boundaries, stale sessions, queued jobs and grant expiry/revocation remain in scope within that organization. Automated cross-tenant isolation tests remain required. This scope decision does not authorize production or customer infrastructure operations.
Latest operational checkpoint (2026-10-22)
The real Platform Apply and HTTP-to-S3 migration acceptance in #90 is now complemented by successful Corporate Application provisioning and same-order recovery from a reported partial Apply failure; see the current acceptance record in #93. The fresh Plan retained 12 existing resources and the user-executed Apply completed with 3 added, 0 changed and 0 destroyed. The user also confirmed Internet/DNS and external SSH after manual firewall NAT/port-forward, VM security-group and guest-user/key configuration. These are positive acceptance results for the tested Corporate instance, not automated firewall-policy or HA/isolation qualification (#101).
Current Dev: #136 at 3d2f0af26e5bac23f23ff7c462cff1cec0f165b7, release 39767149101, Runner pool size 1, dev-skip-assistant-qualification, build_only=false. Deployment/migrations and isolated cold/warm Runner probes passed; the Assistant remains enabled. Live Assistant qualification was intentionally skipped for this unrelated fix and is not claimed by this release.
Two accounts have been exercised in one organization. A second real organization is outside the current acceptance scope by the 2026-10-23 user decision; live revocation and role-boundary acceptance remain outstanding. Public Application cloud acceptance has subsequently succeeded (see the 2026-10-23 checkpoint above). Hard Runner loss, backup/restore, capacity and rollback criteria remain open. #135 has subsequently merged but is not deployed; the current acceptance does not qualify its Automatic IPv4 and identity-completion changes. No production or agent-executed customer Plan/Apply was performed. Earlier queued-merge statements below are historical.
Historical source integration
PR #102 contains the reviewed MVP source integration. Final source: c0e95ac. All 13 final-revision native/Configurator CI checks passed. The configured real models passed a complete 114/114 repeated suite and 13/13 continuous journey; prior failed evidence remains recorded in the PR.
The PR is ready and protected auto-merge is enabled. Required human/code-owner review is still outstanding; repository protections were not weakened or bypassed. Source integration is not an application deployment, customer Apply or production acceptance.
Acceptance checklist
- Maintain current capability/boundary documentation, linked implementation issues and a distinction between source/mock/native and real customer evidence.
- Freeze and verify the final revision through build/unit/browser/PostgreSQL isolation, native OpenTofu contracts and catalogue checks.
- Accept separate users/roles in the available organization, revocation, interrupted Runner and recovery paths; retain automated cross-tenant isolation coverage. A second real organization is deferred to later customer validation and does not block current acceptance.
- Verify PostgreSQL/Secrets Manager/artifact backup/restore, credential rotation and safe operator recovery diagnostics against the acceptance environment.
- Complete capacity, job cleanup, migration rollback/forward recovery and release rollback compatibility acceptance.
- Review CODEOWNERS/main protection and CI/CD transition; Configurator provisioning and deployment are now exclusively manual.
- Prepare a reviewable PR and secret scan with a clean worktree. The sole history scanner result was independently verified as a synthetic mocked-provider password, not a credential.
- Obtain the user decision authorizing source integration; no customer Apply or production creation is implied.
- Complete source integration of feat: integrate Landing Zone Configurator MVP #102: GitHub records it merged on 2026-10-16 as
0835c7a31618a5ffd007000bd3d9bcf4b3e693ee. This records merge completion, not independent verification of every historical review event.
Maintained references
The Configurator README in #102 links the user guide, component diagrams/architecture, MVP scope and development map. CONTRIBUTING describes the cross-component workflow; AGENTS.md references the preserved Terraform style guide. Dated workstation handoffs and historical readiness/runtime ledgers have been removed from the maintained tree and are not current acceptance evidence.
Native CI: https://github.com/stackitcloud/stackit-landing-zone/actions/runs/38898519075
Configurator CI: https://github.com/stackitcloud/stackit-landing-zone/actions/runs/38898519039
Remaining scope
This issue remains open for the operational acceptance above. No production environment was created; production identity, network, backup/recovery, capacity and operator qualification are a separate follow-up. Local/model/CI success does not certify every live customer topology.
Product acceptance continues to depend on #89, #90, #91, #92, #93 and #94. #94 is linked for source-merge closure; the other issues retain their unaccepted live criteria.
Activity
lweberru commentedon Oct 4, 2026
MVP audit 2026-10-04: Updated the description and acceptance checklist against the current implementation. Implemented criteria are checked; remaining requirements are retained. Local tests are not a release or live customer acceptance. This issue stays open until its remaining criteria are met. The first remaining safety fix (#89, backend-free initial plans on existing/partial/legacy state) is now implemented locally: 23/23 isolated PostgreSQL tests and the full check gate (310 unit tests) pass. No customer Apply, API restart, native artifact rebind, commit or merge was performed.
lweberru commentedon Oct 16, 2026
MVP source integration is prepared in #102: all changes are committed/pushed; automatic native/Configurator checks are separated from manual cloud operations; public user/architecture docs and contributor/operator references are separated; CONTRIBUTING and AGENTS.md are added/updated; obsolete workstation/evidence artifacts are removed from the maintained tree. Existing main rules and required reviewer approval are unchanged. The user has authorized integration, but that does not bypass repository review rules.
The PR records exact CI/model evidence and retained failed attempts. The final complete Assistant run is 114/114 plus a 13/13 journey. The final workflow-hardening commit is being verified separately; no deployment, customer Apply or production setup has occurred.
Keeping this operational acceptance issue open. Two-organization/revocation acceptance, real interrupted-runner/migration recovery, backup/restore/rotation and production capacity/rollback/operator qualification remain separate uncompleted requirements. Source integration and green test jobs are not a declaration that those live paths were accepted. Maintained references are the user guide, architecture, MVP scope and development map linked from the Configurator README in #102, rather than the removed dated readiness/runtime ledgers.
lweberru commentedon Oct 16, 2026
Final integration update: all 13 checks on c0e95ac passed. PR #102 is ready, and exact-head protected auto-merge is enabled with the required review still pending. The issue checklist now reflects the completed source/CI/documentation work without marking untested live acceptance complete. #59, #82, #84 and #94 are linked for closure when the PR merges; no deployment or production resource was created.
lweberru commentedon Oct 17, 2026
Development delivery follow-up: #109–#114 are merged. The normal manual release 38986171056 succeeded on exact source
3ab7bfaa92236fc4a4242028a27a901df0c1496d; only the existing protected Development environment was approved. API and Runner current staged droplets were independently verified at that exact source.Exact-run evidence: configured Qwen model 114/114 repeated cases and 13/13 continuous dialogue; 817 unit tests passed (66 environment-dependent skips), 60 isolated PostgreSQL cases, 308 browser cases and pinned native contracts. Final actual-host browser smoke passed 5/5, including direct Plan/Apply URLs and reload, both viewport approval/blocker flows, locale, protected boundaries and same-tab STACKIT login.
The first otherwise successful release exposed real hosted phase-URL 404s hidden by the Vite preview; #114 fixed the explicit API SPA allowlist and added actual-static-serving regression coverage. Earlier model/release/browser failures remain preserved, not replaced by unchanged-source qualification retries.
Apply UI cases use intercepted synthetic APIs, not customer state; they never click Apply and forbid execution writes. The particular customer denial has not been diagnosed, and Firewall/HA Plan-only restrictions remain intact. No customer Apply or production deployment occurred. Source integration/development delivery is now complete; this issue remains open for the separately agreed operational acceptance, recovery and production follow-up.
lweberru commentedon Oct 20, 2026
Dev release blocked by Model Serving transport failures
PRs #119 and #120 are merged. The authorized manual Dev release with
runner_pool_size=1, run 39406372348, used exact sourcefc8ec29141c209769fa905504f83c921e7c22efaand failed qualification before any build, publication, migration or deployment.dns-zone-role-advice, iteration 1: model-phaseconnection_failed, 5591 ms.dns-owner, iteration 2: model-phaseconnection_failed, 3192 ms.platform-public-corporate-networkscase passed both independent repetitions. Continuous dialogue: 13/13 passed./v1/modelschecks from the workstation returned HTTP 200 and listed the configured model. These checks are not inference qualification, do not reproduce the GitHub network path and do not resolve or replace the two failed repetitions.Failed receipts remain attached to the original run. No failed repetition was retried or reclassified, and no oracle, timeout, model or release protection was weakened. Investigating the original transport cause requires runner-side bounded cause diagnostics or corresponding service-side evidence; a speculative client change is not justified by the existing receipts.
The existing Dev public health check still succeeds. The new release has not been deployed; hosted warm handoff/replacement proof remains unverified. No customer Apply or production operation was performed.
lweberru commentedon Oct 20, 2026
Bounded production inference retry implemented in #121
Following the user discussion, #121 adds one additional request for selected transient pre-response network errors, with 250-500 ms jitter inside the original deadline. It preserves cancellation, never replays HTTP/body/semantic failures, and never retries Plan/Apply or persistence. Safe per-attempt codes are retained in production logs and synthetic receipts; no CI-only recovery path or factual oracle change.
Exact source:
8f65ea8928f6e23bf7c8d62a9a7b262de0d59667. Pinned canonical validation: 971 passed, 69 unchanged default-gated tests. Changed-source real-model DNS cases: 16/16, two independent repetitions; continuous dialogue 13/13, both exit 0. No actual transport failures occurred locally; controlled tests verify retry and exhaustion without claiming the original provider/network cause was reproduced. Knowledge SHA:6179c3ad16fe011412c165ed1a0d2d71bff0678b9ba3dc2642464fbc3a41db69.Normal protected squash auto-merge is queued; CODEOWNER review remains required. Original failed runs and receipts remain failed. Dev is not updated yet: after approved integration, a new exact-source manual release with
runner_pool_size=1must still pass full qualification and hosted warm evidence.fix(configurator): bound transient model transport retries (#121)
lweberru commentedon Oct 20, 2026
Next exact-source release: factual setup-routing defect identified
The user merged #121 as
0c5188bf60c332adcbecf5372e913021b298e126. Its tree matched the validated feature tree. The optional extra local full run was cancelled after four passing cases, with no failures, to avoid concurrent inference with the protected release; it is not qualification evidence.New manual Dev release 39412434669,
runner_pool_size=1, attempt 1: 113/114 cases, 13/13 dialogue, no transport-error events.firewall-metro-setup-operationsrepetition 1 omitted the unanswered operations decision. Build, publication, migration and deployment were skipped. The original failed receipt remains unchanged.Draft #122 fixes a reproduced structural cause: general
hafield hints could bypass the dedicated scope/evidence classifier on a broad setup continuation. Four red regressions prove that bypass; the fix preserves explicit narrow native updates and uses null-only evidence when no bounded quote fragments exist. No factual oracle or topic facts changed. Canonical validation is green: 977 passed, 69 unchanged default-gated tests. Real-model Firewall qualification and CI are in progress; review readiness waits for their results. Dev remains on the previous release, without hosted proof of the new warm pool.fix(configurator): preserve broad Firewall setup decisions (#122)
lweberru commentedon Oct 20, 2026
Dev release: qualification blocked; Model Serving maintenance confirmed
PR #122 is merged at
d0fc3d3ef1cb4c405fdfdb95e2a45cfb707b33c8.The fresh manual release run 39417609958
requested
runner_pool_size=1but did not build or deploy:dns-placeholder, repetition 2,upstream_status_403, at2026-10-20T12:04:25Z.The official STACKIT status API
reported AI Model Serving
under_maintenanceat2026-10-20T12:16:36Z:GPU Maintenance Notice Part 2,
in_progress,scheduled until 18:00 CEST / 16:00 UTC. The original notice text says
17:00 CEST instead; actual service recovery matters, not the advertised clock time.
Maintenance confirms possible service interruptions but does not establish
the specific reason for the observed HTTP 403.
Recovery boundary: wait for the component to become operational and relevant
maintenance to complete, then run a fresh complete protected manual release
against unchanged main, with
runner_pool_size=1and unchanged qualificationcriteria. Do not rerun individual failures, erase failed repetitions, retry 403,
switch models or weaken the gate. The failed run remains part of the evidence.
UI reproduction after recovery: Actions > Configurator Release > Run workflow,
branch
main,build_only=false,runner_pool_size=1.Normal protected Dev approval remains required after qualification/build.
No customer infrastructure Apply or production deployment is included.
lweberru commentedon Oct 21, 2026
Live platform acceptance update (2026-10-21)
The operator supplied a successful real saved-plan Apply log and confirmed the completed workflow. The retained partial-bootstrap deployment was resumed through checkpoint review, explicit same-state reconciliation, a new credential profile, a new configuration revision/preparation and a newly reviewed saved Plan. Existing firewall VMs were preserved.
Observed results:
Apply complete! Resources: 13 added, 0 changed, 1 destroyed.The replacement was only the taintedterraform_dataHA bootstrap step, not a firewall VM. Both backup and primary HA configuration commands completed successfully. All three project-qualified management buckets, Object Storage credentials, the audit archive destination, the Observability Starter instance and associated protected secret entries were created. The subsequenttofu init -migrate-statereported successful migration from the HTTP bootstrap backend to S3.Delivered by #128, #129 and #130 through Dev release 39608332090, with runner pool size 1. This was the explicit Dev qualification-exception mode, with the Assistant enabled; it is not production acceptance or a new successful full model qualification. Evidence here is a sanitized summary of operator-provided output, not independent access to customer infrastructure. Customer identifiers, credentials, raw plans and State are deliberately omitted.
A clean follow-up Plan has not been independently established by the supplied log. Actual HA failover, packet paths and audit-event delivery were deferred. Hard Runner loss, interrupted backend migration, backup/restore and multi-tenant acceptance are not proven by this successful run.
The source-merge checklist has also been corrected against GitHub: #102 merged on 2026-10-16. Operational acceptance remains open.
lweberru commentedon Oct 23, 2026
Repeatable local recovery infrastructure and deterministic engine tests
PR #141 adds a digest-pinned Compose recovery lab, isolated from hosted Dev and cloud execution. Actual local selftests with all 66 current SQL migrations passed repeatedly: 51 tables, migration ledger, effective relation ACLs/owners/RLS and function definitions/owners matched after restoration into an independent volume. Six isolation guards also passed. No customer backups or cloud credentials were imported.
The real OpenTofu 1.12.6 controller passed a running-Plan hard stop at a deterministic local state-read barrier, followed by a successful fresh Plan. Its existing-State Apply scenario also passed: after a real intermediate checkpoint and process-group hard kill, a fresh Plan/Apply retained the resource newly created during that interrupted Apply without replacement or ID change.
A separate initially empty local-backend case failed: the observed intermediate checkpoint omitted an already completed resource. #140 retains the synthetic reproduction and investigation; the optional diagnostic fails instead of masking that omission. The passing existing-State test is not acceptance of the first-Apply case.
These results do not establish hosted Runner-loss reconciliation, S3 lock behavior or cloud checkpoint completeness, nor full Secrets Manager/artifact/service restoration. Those criteria remain open. No production, customer Plan/Apply, force-unlock or customer State write occurred. The user separately reported successful Public Plan/Apply with Observability in #93/#95.
test(configurator): add repeatable isolated recovery lab (#141)
fix(assistant): ground DNS completion in saved metadata context (#142)
test(configurator): cover remote State interruption and orphaned locks (
lweberru commentedon Oct 23, 2026
Dev deployment completed successfully: promotion 39943140329 deployed the exact immutable qualified build 39930756409, source
d74689be46b45ffed9981fac844e1f36550c4120, withrunner_pool_size=1. Both source-bound real-model qualification and the complete build passed. The original run stopped at its superseded-branch publication guard after the test-only #143 merge; no deployment occurred in that run. The documented operator backfill verified the exact protected buildpack tag, and explicit artifact promotion retained the original source and checksums. No guard was relaxed.The promotion passed versioned migrations, isolated Runner/binding checks and public route checks. Independent read-only verification confirmed health, STACKIT authentication, protected session/Assistant/recovery APIs and the recovery UI. Assistant and execution remain enabled. No customer Plan/Apply was initiated by this release.
This makes #135 and #142 available in Dev. It does not qualify Automatic IPv4 on a newly source-bound cloud order, hosted interruption/reconciliation, full-service restore or other outstanding acceptance criteria. The user has identified a disposable not-yet-executed order for a future targeted hosted drill; its identifiers remain private.
test(configurator): target hosted Plan interruption safely (#144)
lweberru commentedon Oct 23, 2026
PR #144 is merged and deployed to Dev by release 39951795196, source
28b1d15dab5d2a469789214d75db8791535bc5bb. All applicable CI checks passed. Local complete checks passed (1,052 tests, 71 gated skips), alongside 18 lab/controller safety tests and the real read-only CF observer startup probe.The release used the explicitly selected Dev-only
dev-skip-assistant-qualificationmode because Assistant facts/capabilities were unchanged. Independent live configuration checks confirmed Assistant enabled, execution enabled and Runner pool size 1. Deployment migrations, service/Runner isolation checks and public protected-route checks passed. A separate synthetic task in the deployed isolated Runner template successfully imported and verified the new observation module without starting any customer job or requesting any Runner stop.The hosted Plan stop controller is now available, defaulting to observation. Stop mode needs the exact disposable instance plus a new private durable intent receipt, rejects stale/finished/mismatching observations, and cannot overwrite an ambiguous earlier receipt. A stop request remains explicitly unqualified until actual Runner exit, terminal-job outcome, lock/State reconciliation and fresh-Plan behavior are checked. No hosted interruption has yet been performed.
The proposed empty test order is bound to an older immutable Accelerator revision than the current Runner. Republishing the same template as a new version and creating a separate disposable order is required; existing orders/State are not silently rebound. Customer identifiers remain private. Apply interruption, full-service restore and #140 remain open.
fix(configurator): reduce hosted Plan stop latency (#146)