fix(metadata-core): the artifact forward conversion walks every packages[].manifest body of an option-B artifact (ADR-0130 D4)

.changeset/23027-package-body-conversion.md

@@ -0,0 +1,17 @@
+---
+'@objectstack/metadata-core': patch
+'@objectstack/runtime': patch
+---
+
+fix(metadata-core, runtime): a multi-package artifact built before a retirement boots at `os dev -a` and `os start`. The ADR-0087 forward conversion walks every `packages[].manifest` body, and the runtime's artifact load converts the bundle once, before any reader sees it
+
+Clause-②: no
+
+**What an operator saw.** A multi-package release artifact (ADR-0130 D4: `os build` of a `composeStacks(…, { manifest: 'preserve' })` project) carries its collections in `packages[i].manifest`. Take such an artifact built before a retirement, for example with a dashboard still spelled `refreshInterval`. `os dev -a dist/objectstack.json` and `os start` exited at boot with "`packages[1]` is not a package entry (ADR-0130 D4): manifest.dashboards.0.refreshInterval: `dashboard.refreshInterval` was renamed to `refreshIntervalSeconds`…". The metadata plugin's artifact door refused the same artifact at its strict parse. The ADR-0087 conversion converted that dashboard when it sat in the artifact's own `dashboards`, but it never read a package body, and the runtime read the bundle raw before any conversion ran.
+
+**What an operator sees now.** The same artifact boots at `os dev -a` and at `os start`. Each body's items are served converted (`refreshIntervalSeconds: 300`, valid), each owned by its own package. The boot log names each conversion once, with the body it found it in: `[MetadataPlugin] artifact '…' … converted 2 site(s) forward via ADR-0087 conversion 'dashboard-refresh-interval-to-refresh-interval-seconds' (first at packages[0].manifest.dashboards[0].refreshIntervalSeconds) …`. The artifact file itself is unchanged, so it converts again on every boot until it is rebuilt with `os build`. A body built after the retirement converts nothing and logs nothing. An artifact authored past every retirement is still refused with the tombstone inside its body.
+
+- **`@objectstack/metadata-core`.** `applyArtifactForwardConversions` runs the same conversion over each package body as over the artifact's own collections. It uses the same registry and the same window, decided once from the artifact's own `engines.protocol`; a body's own `engines.protocol` is not read. A notice from a body is located from the artifact root (`packages[1].manifest.…`).
+- **`@objectstack/runtime`.** `loadArtifactBundle` converts the loaded artifact once, before it merges the runtime module and before the stack, `AppPlugin` or the `manifest` service read it. The database resolver reads the artifact file itself, and it runs the same conversion on what it reads. So a multi-package artifact that declares its default datasource in a package body opens that database, not the unified default.
+
+**Unchanged.** An artifact without `packages[]` converts exactly as before, with the same notices, paths and messages, and it is served the same way.

packages/metadata-core/src/artifact-forward-conversion.ts

@@ -111,6 +111,38 @@
  * Cross-major gaps are the protocol *handshake*'s jurisdiction
  * (`checkProtocolCompat`) and refuse before conversion could matter.
  *
+ * ## Reach — the definition's own collections AND every package body
+ *
+ * A multi-package artifact (ADR-0130 D4, option B) carries its collections in
+ * `packages[i].manifest`, each an ASSEMBLED package body
+ * (`AssembledPackageBodySchema`) that the strict parse gates entry by entry —
+ * in `ObjectStackDefinitionSchema` and in `resolveArtifactPackageOrder` alike.
+ * Since the 2026-09-22 addendum to D4 the definition's top level carries no
+ * copy of them. So the same walk runs over each body as a second root: the
+ * same `applyConversions` call, the same registry, the same window. ⛔ Never a
+ * second walk with its own entry list — a body would then answer a different
+ * question than the definition it travels in.
+ *
+ * A body is a stack-shaped root: where its manifest half and its collection
+ * half declare the same key (`objects`, `datasources`, `permissions`), the
+ * collection wins (the precedence table on `AssembledPackageBodySchema`), and
+ * those are exactly the keys the registry walks at a definition's top level.
+ * The one entry that reads a manifest (`manifest-permissions-string-list-removed`)
+ * reaches `packages[i].manifest` from the definition root itself, which runs
+ * first; a body has no `manifest` key for it to read again.
+ *
+ * The window is the ARTIFACT's — decided once from the definition's own
+ * `engines.protocol` — and a body's own `engines.protocol` is not read. The
+ * floor stands in for when the artifact was built (see above), and one
+ * `os build` serializes every body of one artifact together (ADR-0130 D1: one
+ * artifact, one atomic delivery). The same reading gates the boot: the
+ * artifact-boot handshake judges the artifact's own `manifest` (`AppPlugin`,
+ * `assertArtifactProtocolCompatible` in `@objectstack/runtime`). A per-body
+ * window would also split the two copies of one definition that an artifact
+ * built before that addendum carries (top level and body) into two answers.
+ * Notices from a body are located from the definition root
+ * (`packages[1].manifest.dashboards[0].refreshIntervalSeconds`).
+ *
  * ## The one class the window does NOT admit — default flips
  *
  * The window above is a RESCUE: without it an artifact carrying a key the
@@ -193,7 +225,10 @@ export interface ArtifactConversionNotice {
   from: string;
   /** The canonical token/shape it was converted to. */
   to: string;
-  /** Where in the stack it applied, e.g. `permissions[0].objects.crm_ticket.allowPurge`. */
+  /**
+   * Where in the stack it applied, e.g. `permissions[0].objects.crm_ticket.allowPurge`;
+   * inside a package body, from the definition's root (`packages[1].manifest.dashboards[0]…`).
+   */
   path: string;
   /** Derived, human-facing one-liner. */
   message: string;
@@ -500,14 +535,66 @@ export function applyArtifactForwardConversions<T>(
   }
 
   const notices: ArtifactConversionNotice[] = [];
-  const converted = applyConversions(definition as Record<string, unknown>, {
-    includeRetired: true,
-    excludeConversionIds,
-    onNotice: (n) => {
-      notices.push(n);
-      options.onNotice?.(n);
-    },
-  }) as T;
+  // ONE walk — the registry's whole chain, under the one window decided above —
+  // run over every root this definition carries: its own collections first,
+  // then each `packages[i].manifest` body (see "Reach" in the module doc). A
+  // body's notice is located under its package, so a boot log names the body.
+  const replay = (root: Record<string, unknown>, at: string): Record<string, unknown> =>
+    applyConversions(root, {
+      includeRetired: true,
+      excludeConversionIds,
+      onNotice: (n) => {
+        const notice = at === '' ? n : locateUnder(n, at);
+        notices.push(notice);
+        options.onNotice?.(notice);
+      },
+    });
+  const converted = replayPackageBodies(replay(definition as Record<string, unknown>, ''), replay) as T;
 
   return { definition: converted, verdict, authoredFloor, runtimeSpecVersion, notices, replayedRetirements };
 }
+
+function isRecord(value: unknown): value is Record<string, unknown> {
+  return value !== null && typeof value === 'object' && !Array.isArray(value);
+}
+
+/**
+ * Run `replay` over each `packages[i].manifest` body (ADR-0130 D4) of an
+ * already-converted definition. Copy-on-write at both levels: an entry whose
+ * body converts nothing keeps its reference, and the definition keeps its own
+ * when no body converted. A `packages` that is not an array, an entry that is
+ * not an object, and a body that is not an object are left exactly as given —
+ * this never gates; `resolveArtifactPackageOrder` (`@objectstack/core`) is the
+ * refusal for each of those shapes.
+ */
+function replayPackageBodies(
+  definition: Record<string, unknown>,
+  replay: (root: Record<string, unknown>, at: string) => Record<string, unknown>,
+): Record<string, unknown> {
+  const packages = definition.packages;
+  if (!Array.isArray(packages)) return definition;
+  let touched = false;
+  const next = packages.map((entry: unknown, i: number) => {
+    if (!isRecord(entry) || !isRecord(entry.manifest)) return entry;
+    const body = replay(entry.manifest, `packages[${i}].manifest`);
+    if (body === entry.manifest) return entry;
+    touched = true;
+    return { ...entry, manifest: body };
+  });
+  return touched ? { ...definition, packages: next } : definition;
+}
+
+/**
+ * A notice emitted inside a package body, located from the definition's root:
+ * `path` gains the body's prefix, and so does the one place `message` quotes
+ * that path (`applyConversions` writes it as ` at <path>: `). Every other field
+ * is the conversion's own.
+ */
+function locateUnder(notice: ArtifactConversionNotice, at: string): ArtifactConversionNotice {
+  const path = `${at}.${notice.path}`;
+  return {
+    ...notice,
+    path,
+    message: notice.message.replace(` at ${notice.path}: `, () => ` at ${path}: `),
+  };
+}

packages/metadata/src/plugin-artifact-forward-conversion-package-body.test.ts

@@ -0,0 +1,146 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * [#23027] An option-B artifact built before a retirement boots at the
+ * metadata door, and the boot log names the package body that carried it.
+ *
+ * A multi-package artifact (ADR-0130 D4, option B) carries its collections in
+ * `packages[i].manifest`. The door's strict parse (`ObjectStackDefinitionSchema`)
+ * gates every body, so a dashboard still spelled `refreshInterval` inside a body
+ * refused the whole artifact (`invalid_type` at
+ * `packages.1.manifest.dashboards.0.refreshInterval`) while the same dashboard
+ * in the definition's own `dashboards` converted. The conversion now runs the
+ * same walk over each body (`applyArtifactForwardConversions`,
+ * `@objectstack/metadata-core`), where the reach, the window and the whole
+ * step over a body are pinned with injected labels. This file pins the real
+ * door on this tree's installed label.
+ *
+ * The fixture is the shape `os build` writes for `examples/app-multi-package`:
+ * the artifact's own `manifest` is its last package's, both packages declare
+ * `engines.protocol: '^18'`, and every collection sits in a body.
+ */
+
+import { describe, it, expect, vi } from 'vitest';
+import { ALL_CONVERSIONS } from '@objectstack/spec';
+import { resolveInstalledSpecVersion } from '@objectstack/metadata-core';
+import { MetadataPlugin } from './plugin.js';
+
+const SUBJECT = 'dashboard-refresh-interval-to-refresh-interval-seconds';
+const CORE_ID = 'com.example.ob.core';
+const ORDERS_ID = 'com.example.ob.orders';
+
+const core = (range: string) => ({
+    id: CORE_ID, namespace: 'ob', name: 'ob_core', version: '1.0.0', type: 'app', engines: { protocol: range },
+});
+const orders = (range: string) => ({
+    id: ORDERS_ID, namespace: 'ob', name: 'ob_orders', version: '2.0.0', type: 'module', engines: { protocol: range },
+    dependencies: { [CORE_ID]: '^1.0.0' },
+});
+const dashboard = (name: string, refresh: Record<string, number>) => ({ name, label: name, widgets: [], ...refresh });
+
+/** An option-B artifact; `ordersDashboard` adds a second body's dashboard. */
+function artifact(range: string, refresh: Record<string, number>, ordersDashboard = false): any {
+    return {
+        manifest: core(range),
+        packages: [
+            { manifest: { ...orders(range), ...(ordersDashboard ? { dashboards: [dashboard('ob_orders_overview', { refreshInterval: 120 })] } : {}) } },
+            { manifest: { ...core(range), dashboards: [dashboard('ob_overview', refresh)] } },
+        ],
+    };
+}
+
+function fakeCtx() {
+    return {
+        logger: { info: vi.fn(), warn: vi.fn(), error: vi.fn(), debug: vi.fn() },
+        registerService: vi.fn(),
+        getService: vi.fn(() => undefined),
+        trigger: vi.fn(),
+    } as any;
+}
+
+function newPlugin(): any {
+    return new MetadataPlugin({ watch: false, config: { bootstrap: 'lazy' } });
+}
+
+/** The conversion summary lines the door logged, keyed by conversion id. */
+function conversionWarns(ctx: any): Map<string, string[]> {
+    const byId = new Map<string, string[]>();
+    for (const [line] of ctx.logger.warn.mock.calls as [string][]) {
+        const id = /ADR-0087 conversion '([a-z0-9-]+)'/.exec(String(line))?.[1];
+        if (!id) continue;
+        byId.set(id, [...(byId.get(id) ?? []), String(line)]);
+    }
+    return byId;
+}
+
+const triple = (v: string) => v.split(/[.-]/).slice(0, 3).map((n) => Number.parseInt(n, 10));
+function compareVersions(a: string, b: string): number {
+    const [x, y] = [triple(a), triple(b)];
+    for (let i = 0; i < 3; i++) if (x[i] !== y[i]) return x[i]! < y[i]! ? -1 : 1;
+    return 0;
+}
+
+describe('[#23027] artifact door — an option-B artifact built before a retirement boots', () => {
+    it('boots: the body\'s dashboard registers as `refreshIntervalSeconds: 300`, owned by its package', async () => {
+        const plugin = newPlugin();
+        const total = await plugin._parseAndRegisterArtifact(fakeCtx(), artifact('^18', { refreshInterval: 300 }), 'option-b');
+        expect(total).toBeGreaterThan(0);
+
+        const registered = (await plugin.manager.get('dashboard', 'ob_overview')) as any;
+        expect(registered, 'the dashboard registers').toBeDefined();
+        expect(registered.refreshIntervalSeconds).toBe(300);
+        expect(registered).not.toHaveProperty('refreshInterval');
+        expect(registered._packageId).toBe(CORE_ID);
+    });
+
+    it('converts per body: each package\'s dashboard registers converted under its own package', async () => {
+        const plugin = newPlugin();
+        await plugin._parseAndRegisterArtifact(fakeCtx(), artifact('^18', { refreshInterval: 300 }, true), 'option-b-multi');
+
+        const overview = (await plugin.manager.get('dashboard', 'ob_overview')) as any;
+        const ordersOverview = (await plugin.manager.get('dashboard', 'ob_orders_overview')) as any;
+        expect([overview.refreshIntervalSeconds, overview._packageId]).toEqual([300, CORE_ID]);
+        expect([ordersOverview.refreshIntervalSeconds, ordersOverview._packageId]).toEqual([120, ORDERS_ID]);
+        expect(ordersOverview).not.toHaveProperty('refreshInterval');
+    });
+
+    it('logs one summary per conversion, counting every body\'s sites and naming the first body\'s path', async () => {
+        const plugin = newPlugin();
+        const ctx = fakeCtx();
+        await plugin._parseAndRegisterArtifact(ctx, artifact('^18', { refreshInterval: 300 }, true), 'option-b-multi');
+
+        const warns = conversionWarns(ctx);
+        expect([...warns.keys()]).toEqual([SUBJECT]);
+        expect(warns.get(SUBJECT)).toHaveLength(1);
+        const [line] = warns.get(SUBJECT)!;
+        expect(line).toContain('2 site(s)');
+        expect(line).toContain('first at packages[0].manifest.dashboards[0].refreshIntervalSeconds');
+        expect(line).toContain(' at packages[0].manifest.dashboards[0].refreshIntervalSeconds: ');
+    });
+
+    it('the same artifact built after the rename boots with nothing converted and nothing logged', async () => {
+        const plugin = newPlugin();
+        const ctx = fakeCtx();
+        await plugin._parseAndRegisterArtifact(ctx, artifact('^18', { refreshIntervalSeconds: 300 }), 'option-b-current');
+        expect([...conversionWarns(ctx).keys()]).toEqual([]);
+        expect(((await plugin.manager.get('dashboard', 'ob_overview')) as any).refreshIntervalSeconds).toBe(300);
+    });
+
+    it('WINDOW SHUT — an artifact authored past every retirement still meets the tombstone inside its body', async () => {
+        // The first `x.y.z` past the installed label and every `retiredAfter`:
+        // a floor no window opens for, whatever label the version pass sets.
+        const installed = resolveInstalledSpecVersion();
+        expect(installed).toMatch(/^\d+\.\d+\.\d+/);
+        const stamps = [installed!, ...ALL_CONVERSIONS.flatMap((c) => (c.retiredFromLoadPath === true ? [c.retiredAfter] : []))];
+        const top = [...stamps].sort(compareVersions)[stamps.length - 1]!;
+        const [major, minor, patch] = triple(top);
+        const current = `^${major}.${minor}.${patch! + 1}`;
+
+        const refusal = await newPlugin()
+            ._parseAndRegisterArtifact(fakeCtx(), artifact(current, { refreshInterval: 300 }), 'option-b-current-floor')
+            .then(() => undefined, (e: any) => e);
+        expect(refusal, 'the strict parse refuses').toBeDefined();
+        expect((refusal.issues as { path: (string | number)[] }[]).map((i) => i.path.join('.')))
+            .toEqual(['packages.1.manifest.dashboards.0.refreshInterval']);
+    });
+});

packages/runtime/src/app-plugin.ts

@@ -498,11 +498,13 @@ export class AppPlugin implements Plugin {
         // served the artifact door's converted copy of it, and what handed the
         // email loader a template it refuses to send.
         //
-        // A bundle carrying `packages[]` is handed over as authored, as before:
-        // the `manifest` service registers its package BODIES, which no
-        // conversion reaches (the ADR-0087 pass walks the definition's own
-        // collections, at the artifact door too), and `forwardConverted` would
-        // resolve `packages[]` here, ahead of the `manifest` service (#15292).
+        // A bundle carrying `packages[]` is handed over as it arrived, as before:
+        // `forwardConverted` would resolve `packages[]` here, ahead of the
+        // `manifest` service (#15292). Its package BODIES are already converted
+        // when the bundle came through `loadArtifactBundle`, which runs the
+        // ADR-0087 pass once over every body at load (#23027); a bundle handed
+        // to this plugin by any other route reaches the `manifest` service as
+        // authored.
         const servicePayload = (this.bundle as { packages?: unknown }).packages === undefined
             ? this.forwardConverted(ctx).definition
             : this.bundle.manifest