Skip to content

fix(core): a bootstrap that fails after every plugin started rolls them back too - #23664

Merged
objectstack-fleet[bot] merged 5 commits into
mainfrom
claude/issue-23644-failed-boot-teardown
Oct 15, 2026
Merged

objectstack-fleet[bot] merged 5 commits into
mainfrom
claude/issue-23644-failed-boot-teardown

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Fixes #23644

Clause-②: no

ObjectKernel.bootstrap() rolled back started plugins only in the Phase 2 branch, for a plugin whose start() failed. A boot refused later (the system-requirement check, or a kernel:ready, kernel:bootstrapped or kernel:listening handler that throws) fell into the outer catch, which marks the kernel stopped, releases the signal listeners and rethrows. shutdown() on a stopped kernel is a no-op, so no plugin's destroy() ran and a driver connected in start() stayed connected. The live case is the dogfood refused boot in organization-ownership-boot-refusal-order, whose own docblock records the stale handle.

What changed

packages/core/src/kernel.ts: the outer catch of bootstrap() now runs the same rollbackStartedPlugins() the Phase 2 branch runs, so a failed boot destroys the plugins that started, in reverse start order, at whatever phase it fails. Its existing duties are unchanged (state stopped, release the listeners, rethrow the original error). The catch's comment and the rollbackOnFailure TSDoc now state the new truth.

The rollback runs when rollbackOnFailure && state === 'running' && startedPlugins.size > 0:

  • rollbackOnFailure is the Phase 2 branch's own gate, so false keeps opting out of both.
  • startedPlugins.size > 0 keeps a Phase 1 failure silent (nothing started) and makes the catch a no-op after the Phase 2 branch rolled back (rollbackStartedPlugins() clears the set), so a failing start() still rolls back exactly once.
  • state === 'running', then stopping for the duration. Two lines beyond the card's text, which I took so this change does not add a double destroy: a shutdown() that began mid-boot already owns the destroys (it destroys every plugin and the set is not cleared), and a shutdown() or signal arriving during the rollback is absorbed by the stopping state exactly as during a drain. Each has its own pin and its own ablation below; both come out cleanly if the seat prefers the minimal form.
  • A destroy() that throws is handled by what the Phase 2 rollback already does: logged at error as Rollback failed for NAME, the loop continues, and the boot error is rethrown unchanged.

Not changed: the Phase 2 branch, shutdown(), LiteKernel, the lifecycle contract, the dogfood file, packages/runtime, packages/verify.

Fork check (card: stop if the contract forbids it or destroy() is unsafe after kernel:ready)

  • packages/spec/src/contracts/plugin-lifecycle-events.ts says nothing about a failed boot and does not forbid destroying plugins after one; content/docs/protocol/kernel/lifecycle.mdx already names destroy() "as rollback for plugins already started when a later plugin's start() fails" and the bootPhaseHooks TSDoc already says that a destroy() which assumes its boot-phase handler ran "must tolerate that it did not".
  • Spot-read destroy() of the plugins that register a boot-phase hook: HonoHttpServer.close() returns when it never listened, DefaultDatasourcePlugin.destroy is best-effort around disconnect, AuthPlugin.destroy only nulls its manager, the job and queue services use optional chaining. No plugin found unsafe after a partial boot.

Pins (packages/core/src/kernel.failed-boot-rollback.test.ts, 21 tests)

  • Tears down.
    • A throw at kernel:ready, kernel:bootstrapped, kernel:listening and at the system-requirement check each destroys the started plugins in reverse start order and disconnects the fake driver connected in start() (exact sequence, so a double destroy reads red).
    • A destroy() that throws during the rollback neither replaces the boot error (identity) nor stops the plugin started before it.
    • The rollback runs with the kernel stopping, and a shutdown() called from inside a destroy() is absorbed.
  • Still one contract.
    • CONTROL: the Phase 2 rollback (a start() that throws) destroys each started plugin once, in reverse order, and never the plugin whose start() threw.
    • shutdown() after a failure at each of the four sites destroys nothing more.
    • The rejection at each hook site is the original error object, and the readiness check rejects with its own.
    • rollbackOnFailure: false leaves the plugins and the driver untouched at each site.
    • A shutdown() that began mid-boot owns the destroys: the failed boot adds none.
    • A Phase 1 failure destroys nothing.

Ablation (committed fix first; each mutation through scripts/ablation-replace.mjs, which proves the anchor hit, the blob change and the restore by blob equal to HEAD and an empty git diff HEAD)

The test imports ./kernel by relative path, so there is no dist/ leg to rebuild.

Mutation (one at a time) Red Green
delete the new await this.rollbackStartedPlugins() 6: the four per-site tear-down pins, the throwing-destroy() pin, the stopping pin 15: the control, shutdown() after failure x4, original rejection x4, rollbackOnFailure: false x4, mid-boot shutdown(), Phase 1
remove state === 'running' 1: mid-boot shutdown() owns the destroys 20
remove the rollbackOnFailure gate 4: rollbackOnFailure: false x4 17
remove this.state = 'stopping' 1: the stopping pin 20
remove this.startedPlugins.clear() from rollbackStartedPlugins 1: the Phase 2 control (a second rollback would double destroy) 20

The observed direction is the usual one (red). Direction is not a template here: the first row's 15 greens are by design, each pins what a failed boot must NOT do and holds with or without the new call.

Verification (head b2a5be7, origin/main merged at 286eb9e)

  • pnpm --filter @objectstack/core typecheck: exit 0 (tsc, the examples project, check:test-typecheck OK with the 4 pinned legacy errors unchanged).
  • @objectstack/core tests: local project 101 files / 2494 tests passed; repo project 6 files / 83 tests passed.
  • eslint (--no-inline-config) on the two changed source files: 0 errors, 0 warnings; 2 files read from --format json; type-aware linting is not enabled (parserOptions carries no project), so this diff cannot move a verdict on an untouched file.
  • Gates from node scripts/pm/dispatch-gates.mjs --commands (63), the artifact-roster block (53) and the wide-population block (11), run after a full turbo run build: 123 of 127 exit 0. The four that exit non-zero need a PR number, a PR body or a workflow event payload (check-closing-target-claim, check-partof-closing-keyword, check-single-claim-paths, pm/check-governed-queue-guard): NOT MEASURED locally, their self-tests exit 0. Two gates exited 3 (prerequisite: no dist/) on the first pass and exit 0 after the build.
  • The live case, read-only: organization-ownership-boot-refusal-order.dogfood.test.ts run against the rebuilt core: 4 of 4 passed, and its refused boot now logs Bootstrap failed after plugins started — rolling back started plugins... once and no Rollback failed for.

Acceptance notes

  • Docs understate, still true. content/docs/protocol/kernel/lifecycle.mdx (the destroy() comment and the rollbackOnFailure: true bullet) describes only the failing-start() rollback. Outside this card's file surface; not edited.
  • No timeout on the rollback. rollbackStartedPlugins() awaits each destroy() with no deadline (shutdown() has shutdownTimeout), so a destroy() that never settles holds bootstrap() open. That was already true for the Phase 2 rollback and is now true for a late failure too. Not probed, not filed.
  • A plugin with no start() is never in startedPlugins, so neither rollback destroys it (for example one that connects in init()). Existing contract of "started", unchanged.
  • LiteKernel has no rollback at all (not on a failing start(), not on a late failure; its destroy() returns early once stopped). A different contract from the one this card extends; not read as a defect, not probed.
  • Idempotent teardown on the host side. A host that closed the leaked resource by hand after a refused boot now closes it a second time; the changeset says so.

Generated by Claude Code

@github-actions github-actions Bot added size/m documentation Improvements or additions to documentation tests tooling labels Oct 15, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

1 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run.

What this run could not see

Coarse fallback — 27 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 1ced0f82bd93260b660e6f448c7dc5952bc82371 → packageMentionDocs.

Which tree this was computed on

This run read content/docs from 8670c435508804309246dc7a565c05601b37fc95 — the merge of head b2a5be7230d471d9d0d025aab7c698dbbc8e055a into base 1ced0f82bd93260b660e6f448c7dc5952bc82371, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 8670c435508804309246dc7a565c05601b37fc95 && git checkout 8670c435508804309246dc7a565c05601b37fc95
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 1ced0f82bd93260b660e6f448c7dc5952bc82371 b2a5be7230d471d9d0d025aab7c698dbbc8e055a && git checkout -B drift-repro 1ced0f82bd93260b660e6f448c7dc5952bc82371 && git merge --no-ff b2a5be7230d471d9d0d025aab7c698dbbc8e055a

node scripts/docs-audit/affected-docs.mjs --json 1ced0f82bd93260b660e6f448c7dc5952bc82371

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review October 15, 2026 11:55
@objectstack-fleet
objectstack-fleet Bot enabled auto-merge October 15, 2026 11:55
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Oct 15, 2026
Merged via the queue into main with commit 8044f4d Oct 15, 2026
41 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/issue-23644-failed-boot-teardown branch October 15, 2026 12:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/m tests tooling

Projects

None yet

2 participants