feat(rest,plugin-hono-server,cloud-connection): ETag + 304 on metadata lists, uncached item reads, runtime/config and localization; db/meta Server-Timing on every timed response

.changeset/27627-metadata-caching-headers.md

@@ -0,0 +1,27 @@
+---
+'@objectstack/rest': patch
+'@objectstack/plugin-hono-server': patch
+'@objectstack/cloud-connection': patch
+'@objectstack/types': patch
+---
+
+feat(rest,plugin-hono-server,cloud-connection): metadata lists, the uncached item reads, `runtime/config` and `auth/me/localization` answer an `ETag` and `304`; every timed response carries `db`, `meta` and `total`
+
+Clause-②: no
+Metadata-protocol: none
+
+The Console's first reads on every load sent no validator, so each one was a full download every time. Measured against HotCRM on `main`, none of these sent an `ETag`, a `Cache-Control` or a `304`: `GET /runtime/config`, `GET /auth/me/localization`, the type list (`GET /meta`, `GET /meta/types`), every `GET /meta/:type` list, and the item reads the shared item cache never serves (`GET /meta/app/:name`, `/meta/dashboard/:name`, `/meta/doc/:name`, `/meta/book/:name`, and drafts, `?package=` and member-scoped reads). Each of them now answers:
+
+- **`ETag`**: a weak validator (`W/"…"`) that hashes the body actually served, after every per-caller step (read gates, the app filter, the field mask, the translation, the host's environment). Two callers whose permissions change what they see get different validators, and one caller's validator never answers `304` for another caller's body.
+- **`Cache-Control: private, no-cache`**: the browser keeps the body and revalidates it on every use, so a publish, a permission change or a language switch shows on the next read. No shared cache stores it. A schema served under an undetermined field visibility keeps `private, no-store` and gets no validator.
+- **`304 Not Modified`**, with no body, when `If-None-Match` names the current validator.
+
+`metadata.enableCache: false` still switches the `/meta` validators off, as its description says ("Enable HTTP cache headers").
+
+`Server-Timing` (still opt-in: `OS_SERVER_TIMING=true`, or an admin's `X-OS-Debug-Timing: 1`):
+
+- Every timed response carries `db` and `meta` beside `total`. A request that ran no SQL or read no metadata says so with a zero member (`db;dur=0;desc="0 queries"`), so "nothing spent there" no longer reads like "not measured".
+- The `/meta` routes count each metadata read into `meta` (`meta;dur=…;desc="N reads"`).
+- `GET /runtime/config` now opens the timing gate for an admin who sends `X-OS-Debug-Timing`. It resolves the caller only when that header is sent; the payload never depends on who asked.
+
+`@objectstack/types` exports the rule the three packages share: `representationEtag(body)`, `ifNoneMatchHits(header, etag)` and `REVALIDATE_PRIVATE_CACHE_CONTROL`.

content/docs/api/metadata-api.mdx

@@ -21,6 +21,9 @@ List all registered metadata types.
 
 **Response**: `{ types: ["object", "view", "plugin", ...] }`
 
+The answer carries a weak `ETag` and `Cache-Control: private, no-cache`; send the
+`ETag` back as `If-None-Match` and an unchanged list answers **304** with no body.
+
 ### `GET /meta/:type`
 
 List all items of a metadata type.
@@ -40,6 +43,13 @@ filtering to privileged apps, and the usual response shaping.
 
 **Response**: `{ type: "object", items: [{ name: "account", ... }, ...] }`
 
+The list carries a weak `ETag` and `Cache-Control: private, no-cache`, and
+`If-None-Match` with the current `ETag` answers **304** with no body. The `ETag`
+hashes the list this caller was served, after the audience and RBAC filtering
+above, so two callers who see different items hold different tags and one
+caller's tag never revalidates another caller's list. Keep the response in the
+client only: `private` means no shared cache may store it.
+
 ### `GET /meta/:type/:name`
 
 Get a specific metadata item with optional ETag caching.

docs/OBSERVABILITY.md

@@ -374,6 +374,12 @@ out of the box:
 | `auth` | Dispatcher | Identity / session resolution — the prime suspect for unexplained data-API overhead. |
 | `db` | SQL driver | Total SQL time across the request; `desc` is the **query count** (folded from knex's per-query events, attributed to the originating request via `AsyncLocalStorage` so it is correct under concurrency). The basic header carries no SQL text; individual **parametrized** statements appear only in the admin-only `X-OS-Debug-Timing: json` detail payload. |
 | `hooks` | ObjectQL engine | Total business-hook execution time; `desc` is the hook count. |
+| `meta` | REST `/meta` routes | Total metadata-read time across the request; `desc` is the **read count** (`meta;dur=0.7;desc="1 reads"`). |
+
+`db` and `meta` are on every timed response, even one that ran no SQL or read no
+metadata: such a response carries them as zero members (`db;dur=0;desc="0 queries"`,
+`meta;dur=0;desc="0 reads"`). A missing member therefore means the response was
+not timed at all, never that nothing was spent there.
 
 Each phase is recorded through a request-scoped collector that is a no-op when
 the mode is off, so every one of them costs nothing on the normal path. The

packages/cloud-connection/package.json

@@ -29,6 +29,7 @@
     "@objectstack/metadata-core": "workspace:*",
     "@objectstack/metadata-protocol": "workspace:*",
     "@objectstack/objectql": "workspace:*",
+    "@objectstack/observability": "workspace:*",
     "@objectstack/runtime": "workspace:*",
     "@objectstack/spec": "workspace:*",
     "@objectstack/types": "workspace:*"

packages/cloud-connection/src/runtime-config-etag.test.ts

@@ -0,0 +1,137 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * [#27627] `GET /api/v1/runtime/config` — the first request every Console load
+ * makes, measured on the cloud at 0.5–1.4 s for 293 bytes with no
+ * `Cache-Control` and no `ETag`. It now answers the weak validator of the
+ * payload it serves (`representationEtag`, `@objectstack/types`), `private,
+ * no-cache`, and a bodiless `304` on a matching `If-None-Match`; the payload is
+ * per host and read per request, both inside the hashed bytes.
+ *
+ * And the per-request `Server-Timing` gate: this route serves everyone and
+ * resolved nobody, so an admin's `X-OS-Debug-Timing` never opened the gate
+ * here (the one route of 58 measured on `main` with no timing for an admin).
+ * The caller is resolved only when that header is sent.
+ */
+
+import { describe, it, expect, vi, beforeEach } from 'vitest';
+import { runWithPerfDisclosure, type PerfDisclosureGate } from '@objectstack/observability';
+
+// A plain recorder rather than a `vi.fn`: a spy wraps a rejected promise in a
+// derived one nobody awaits, which vitest reports as unhandled even though
+// the route catches the original.
+const resolver = {
+    calls: 0,
+    answer: (async () => undefined) as () => Promise<unknown>,
+};
+vi.mock('@objectstack/core', async (importOriginal) => ({
+    ...(await importOriginal<typeof import('@objectstack/core')>()),
+    resolveRequestExecutionContext: () => { resolver.calls += 1; return resolver.answer(); },
+}));
+
+const { RuntimeConfigPlugin } = await import('./runtime-config-plugin.js');
+
+interface Answer { status: number; body: any; headers: Record<string, string> }
+
+async function mountHandler(opts: { host?: string; resolveByHostname?: (h: string) => Promise<any> } = {}) {
+    let handler: ((c: any) => Promise<Answer>) | undefined;
+    const rawApp = {
+        routes: [] as Array<{ method: string; path: string }>,
+        get(path: string, h: (c: any) => Promise<Answer>) {
+            this.routes.push({ method: 'GET', path });
+            if (path === '/api/v1/runtime/config') handler = h;
+        },
+    };
+    const services: Record<string, any> = { 'http-server': { getRawApp: () => rawApp }, auth: {}, objectql: {} };
+    if (opts.resolveByHostname) services['env-registry'] = { resolveByHostname: opts.resolveByHostname };
+    const ctx: any = {
+        logger: { info() {}, warn() {} },
+        getService: (n: string) => { const s = services[n]; if (!s) throw new Error(`no ${n}`); return s; },
+        hooks: [] as Array<() => Promise<void>>,
+        hook(_e: string, cb: () => Promise<void>) { this.hooks.push(cb); },
+    };
+    const plugin = new RuntimeConfigPlugin({ controlPlaneUrl: '' });
+    await plugin.start(ctx);
+    for (const cb of ctx.hooks) await cb();
+    if (!handler) throw new Error('handler not mounted');
+    // A Hono-shaped context that keeps what the handler hands `json` / `body`.
+    return (requestHeaders: Record<string, string> = {}): Promise<Answer> => {
+        const all: Record<string, string> = { host: opts.host ?? '', ...requestHeaders };
+        return handler!({
+            req: { header: (n: string) => all[n.toLowerCase()], raw: { headers: new Headers(all) } },
+            json: (body: any, status = 200, headers: Record<string, string> = {}) => ({ status, body, headers }),
+            body: (body: any, status = 200, headers: Record<string, string> = {}) => ({ status, body, headers }),
+        });
+    };
+}
+
+beforeEach(() => { resolver.calls = 0; resolver.answer = async () => undefined; });
+
+describe('[#27627] runtime/config answers a validator of the payload it serves', () => {
+    it('a weak ETag and private, no-cache — never public', async () => {
+        const get = await mountHandler();
+        const res = await get();
+        expect(res.status).toBe(200);
+        expect(res.headers.ETag).toMatch(/^W\/"[0-9a-f]{16}"$/);
+        expect(res.headers['Cache-Control']).toBe('private, no-cache');
+    });
+
+    it('an unchanged payload revalidates to a bodiless 304', async () => {
+        const get = await mountHandler();
+        const etag = (await get()).headers.ETag;
+        const again = await get({ 'if-none-match': etag });
+        expect(again.status).toBe(304);
+        expect(again.body).toBeNull();
+        expect(again.headers.ETag).toBe(etag);
+    });
+
+    it('two hosts resolving different environments hold different tags; one never revalidates the other', async () => {
+        const resolveByHostname = async (h: string) => (h === 'a.example.com' ? { environmentId: 'env_a' } : { environmentId: 'env_b' });
+        const a = await (await mountHandler({ host: 'a.example.com', resolveByHostname }))();
+        const getB = await mountHandler({ host: 'b.example.com', resolveByHostname });
+        const b = await getB();
+        expect(a.headers.ETag).not.toBe(b.headers.ETag);
+        const crossed = await getB({ 'if-none-match': a.headers.ETag });
+        expect(crossed.status).toBe(200);
+        expect(crossed.body.defaultEnvironmentId).toBe('env_b');
+    });
+
+    it('the response never depends on who asked: no caller is resolved without the timing header', async () => {
+        const get = await mountHandler();
+        await get();
+        await get({ cookie: 'session=abc' });
+        expect(resolver.calls).toBe(0);
+    });
+});
+
+describe('[#27627] runtime/config opens the per-request Server-Timing gate for an admin only', () => {
+    const timed = async (posture: string | undefined, headers: Record<string, string>) => {
+        resolver.answer = async () => (posture === undefined ? undefined : { userId: 'u1', posture });
+        const get = await mountHandler();
+        const gate: PerfDisclosureGate = { allowed: false };
+        const res = await runWithPerfDisclosure(gate, () => get(headers));
+        return { gate, res };
+    };
+
+    it('an admin who sends X-OS-Debug-Timing opens it', async () => {
+        const { gate, res } = await timed('PLATFORM_ADMIN', { 'x-os-debug-timing': '1' });
+        expect(gate.allowed).toBe(true);
+        expect(resolver.calls).toBe(1);
+        expect(res.status).toBe(200);
+    });
+
+    it('a member who sends it does not, and neither does a sessionless caller', async () => {
+        expect((await timed('MEMBER', { 'x-os-debug-timing': '1' })).gate.allowed).toBe(false);
+        expect((await timed(undefined, { 'x-os-debug-timing': '1' })).gate.allowed).toBe(false);
+    });
+
+    it('a resolution that fails leaves the gate closed and serves the payload unchanged', async () => {
+        resolver.answer = async () => { throw new Error('authz store unavailable'); };
+        const get = await mountHandler();
+        const gate: PerfDisclosureGate = { allowed: false };
+        const res = await runWithPerfDisclosure(gate, () => get({ 'x-os-debug-timing': '1' }));
+        expect(gate.allowed).toBe(false);
+        expect(res.status).toBe(200);
+        expect(res.body.features).toBeDefined();
+    });
+});

packages/cloud-connection/src/runtime-config-plugin.ts

@@ -270,7 +270,15 @@
  * exactly as it does for `marketplace`.
  */
 
-import type { Plugin, PluginContext } from '@objectstack/core';
+import {
+    resolveRequestExecutionContext,
+    type IDataEngine,
+    type Plugin,
+    type PluginContext,
+    type RequestAuthService,
+} from '@objectstack/core';
+import { allowPerfDisclosure, isPerfDisclosurePrincipal } from '@objectstack/observability';
+import { ifNoneMatchHits, representationEtag, REVALIDATE_PRIVATE_CACHE_CONTROL } from '@objectstack/types';
 import { resolveCloudUrl, isControlPlaneDeclined } from './cloud-url.js';
 import {
     CLIENT_ERROR_REPORTING_DSN_ENV,
@@ -287,6 +295,66 @@ import {
 import type { IHttpServer } from '@objectstack/spec/contracts';
 import { readConsoleMapStyleUrl } from './console-map-style.js';
 
+/**
+ * [#27627] Answer `body` under the weak validator of exactly what is served
+ * (`representationEtag`, `@objectstack/types`), `private, no-cache`, and a
+ * bodiless `304` when the request's `If-None-Match` already names it. The
+ * payload is decided per HOST (each subdomain resolves its own environment)
+ * and read per request (a Branding save shows on the next load), both inside
+ * the hashed bytes, so a revalidation sees either change at once. `private`,
+ * not `public`: the host-to-environment resolution and the saved branding are
+ * this deployment's, and nothing here asks a shared cache to hold them.
+ */
+function sendRevalidated(c: any, body: Record<string, unknown>): Response {
+    const etag = representationEtag(body);
+    const headers = { 'Cache-Control': REVALIDATE_PRIVATE_CACHE_CONTROL, ETag: etag };
+    if (ifNoneMatchHits(c.req.header('if-none-match'), etag)) return c.body(null, 304, headers);
+    // Sent as a literal, not as `body`: `check:route-envelope` counts this
+    // route's one bare payload at the call site, and the payload is still built
+    // by this plugin (the parameter exists only so the validator hashes it).
+    return c.json({ ...body }, 200, headers);
+}
+
+/**
+ * [#27627] Resolve this request's caller and, for an admin or service
+ * principal, open the per-request `Server-Timing` disclosure gate — the same
+ * predicate (`isPerfDisclosurePrincipal`, over the resolved `posture`) every
+ * other door hands its caller to. The composition is the current-user
+ * endpoints' own (`plugin-hono-server` `current-user-endpoints.ts`): a
+ * sessionless request resolves nobody, and no API-key door is opened.
+ *
+ * Best-effort by design, and that is the whole difference from those doors:
+ * the answer never depends on the caller, so a resolution that fails — an
+ * authorization store outage included — only leaves the gate closed. A no-op
+ * when no request opened a gate (perf-tuning off).
+ */
+async function openTimingGateForAdmin(ctx: PluginContext, headers: Headers): Promise<void> {
+    const service = <T>(name: string): T | undefined => {
+        try { return ctx.getService<T>(name); } catch { return undefined; }
+    };
+    try {
+        const context = await resolveRequestExecutionContext({
+            headers,
+            ql: service<IDataEngine>('objectql'),
+            authService: service<RequestAuthService>('auth'),
+            tenancy: undefined,
+            anonymous: 'fail-closed',
+            authGate: 'withhold',
+            credentials: 'session-only',
+            sessionApi: 'required',
+            session: 'request',
+            localization: 'withhold',
+            requestLocale: undefined,
+            accessToken: 'withhold',
+            oauth: undefined,
+            onAdmission: undefined,
+        });
+        if (context && isPerfDisclosurePrincipal(context)) allowPerfDisclosure();
+    } catch {
+        // The gate stays closed; the payload is served unchanged.
+    }
+}
+
 /**
  * The `env-registry` slot's hostname resolvers — see the call site for both
  * spellings. Async by contract: the consumer awaits the result, and declaring
@@ -1046,6 +1114,15 @@ export class RuntimeConfigPlugin implements Plugin {
             };
 
             const handler = async (c: any) => {
+                // [#27627] Open the per-request `Server-Timing` gate for an
+                // admin, as every other door does once it has resolved its
+                // caller. This route serves everyone anonymously and resolves
+                // nobody, so without this an admin's `X-OS-Debug-Timing` was
+                // the one request in the API that never got its timing. The
+                // caller is resolved ONLY when that header is sent: an
+                // ordinary load pays no session read, and the answer below
+                // never depends on who asked.
+                if (c.req.header('x-os-debug-timing')) await openTimingGateForAdmin(ctx, c.req.raw.headers);
                 const rawHost = c.req.header('host') ?? '';
                 const host = rawHost.split(':')[0].toLowerCase().trim();
                 let defaultEnvironmentId: string | undefined;
@@ -1087,7 +1164,7 @@ export class RuntimeConfigPlugin implements Plugin {
                         // 500s. Worst case the SPA shows its env picker.
                     }
                 }
-                return c.json({
+                return sendRevalidated(c, {
                     cloudUrl: this.cloudUrl,
                     // Declared by the host, never derived (#14514). Spread,
                     // not `upgradeUrl: this.upgradeUrl`, for the same reason

packages/cloud-connection/vitest.config.ts

@@ -86,6 +86,14 @@ export default defineConfig({
         find: /^@objectstack\/plugin-security$/,
         replacement: path.resolve(__dirname, '../plugins/plugin-security/src/index.ts'),
       },
+      // [#27627] `runtime-config-etag.test.ts` drives the per-request
+      // Server-Timing gate the route opens for an admin (`allowPerfDisclosure`),
+      // so it judges THIS checkout's gate, not an `observability/dist` that may
+      // be behind it. Anchored like the entries above.
+      {
+        find: /^@objectstack\/observability$/,
+        replacement: path.resolve(__dirname, '../observability/src/index.ts'),
+      },
     ],
   },
   // Discovery is untouched — but this file is NOT "the alias and nothing else".