Repository navigation
perf(metadata-protocol): one GET /api/v1/data read makes one sys_metadata read of its object, not five - #24391
Conversation
… reads from one store read of both spellings Co-authored-by: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fe2h3xpvRmGq9TqyCTV9JC
…age-less binding the read already selected Co-authored-by: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fe2h3xpvRmGq9TqyCTV9JC
…llings read Co-authored-by: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fe2h3xpvRmGq9TqyCTV9JC
…'s both-spellings read Co-authored-by: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fe2h3xpvRmGq9TqyCTV9JC
…tions against the store's own answers Co-authored-by: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fe2h3xpvRmGq9TqyCTV9JC
…per by-name read Co-authored-by: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fe2h3xpvRmGq9TqyCTV9JC
…ta-read-overlay-row-once
…icate; the double ledger records both new pins Co-authored-by: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fe2h3xpvRmGq9TqyCTV9JC
📓 Docs Drift CheckThis PR changes 1 package(s): 15 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
⛔ 8 release-owned page(s) also name something this change touched. These are read-only:
What this run could not see
Coarse fallback — 11 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin d55981bf48391a5b6ce7e8b34d1840acb23bf441 && git checkout d55981bf48391a5b6ce7e8b34d1840acb23bf441
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin cf16a0d18aedbe58e9bfc0d7bc140fa660465eaf f95842334f57bfe991269086bb23a4db0363624c && git checkout -B drift-repro cf16a0d18aedbe58e9bfc0d7bc140fa660465eaf && git merge --no-ff f95842334f57bfe991269086bb23a4db0363624c
node scripts/docs-audit/affected-docs.mjs --json cf16a0d18aedbe58e9bfc0d7bc140fa660465eaf
|
Contract reviewServed-tier: Read for this record: card #24371's body and all four comments (triage 6252555847, claim 6252699680, dispatch 6252724947, ① Derived judgmentsThe change. At base, Selection by the same
So every spelling pair the two maps know is covered, not only Drafts are untouched by the change:
Each answer, case by case:
Cache: none.
Ordering and case each have a residue. Neither is a contract breach.
5 → 1 count: holds for the card's case.
The flipped
The new differential pin ( The doubles: 42 sampled, every one.
Ledger. Gates (the check-runs on this head, read at 20:20Z). All 40 are complete: 34
② Semver level
The prose face is accurate, apart from F1's scope. That includes "Every real driver (sql, memory, mongodb) already supports ③ Boundary flagsDeviations:
Open question 1 (the stored-row second read): A, accepted.
Open question 2 (the shape of the single read): A, accepted.
F1 (this review): an acceptance note, not a card. It is a move toward the case-exact contract, and the defect class does not apply. Implemented-by: VERDICT: PASS |
席位核对 ·
|
| Item | Reading |
|---|---|
| Shape | Draft PR → main. The body opens Fixes #24371. 48 files, +839 / −91. Not governed (check-governed-merges.mjs --pr 24391); 930 changed lines, under the human-merge threshold. |
| Scope | One runtime file, packages/metadata-protocol/src/protocol.ts: readMetaItem reads the item's active rows once (readItemActiveRows, type $in both spellings), and the three lookups select from that read by the where each used to send (rowMatchesOverlayWhere). Two new pins, the re-aimed protocol-meta.test.ts, two gate-written ledger rows, and about 40 test doubles taught $in. The doubles' files in packages/rest and packages/runtime are declared to domain:cli (6254472332, 6254478160); the objectql ones are in this lane. One changeset, patch. |
| Seat's own read | At the head, readItemActiveRows is the one read, with type: { $in: [type, other] }, name, state: 'active'. findServedOverlayRow and overlayLockLayerAt select from activeRows when it is passed. Only readMetaItem reads into it, and it is a local of that call. |
| Measured | The trace: all five reads came from the one getMetaItem call the /data exposure gate makes. On Postgres behind a 50 ms round trip, a GET /api/v1/data/OBJECT went from a median of 551 ms with 5 sys_metadata statements to 340 ms with 1, with identical bodies once ids and timestamps are masked. |
| Review face | Clause-②: no. The seat's review of record at TIER_DEFAULT, 6254708368 on this head, is PASS. The seat verified its central claims at the head (the row above). |
| Declaration | Clause-②: no on the PR body and the changeset; nothing published widens or narrows. The changeset is patch: only private members move. |
| Pins and ablation | The door pin counts exactly one find for a list and a get of a code object, find + findOne for a stored object, and checks that an edited row is seen by the next request. The differential pin checks every lookup from the one read against the store's own answer, over 76 object and 584 email_template row arrangements. Ablating the one read turned the count rows red; the restore read back as the HEAD blob. |
| Gates | Dev at f95842334f: 79 derived, 79 run, 0 NOT MEASURED. Typecheck exit 0 for metadata-protocol, rest, objectql and runtime. |
| CI | At this read: 40 check-runs on f95842334f, 34 success, 6 skipped, 0 failed. The skips are the rostered path and opt-in skips. Test Core, every Type Check lane, Lint & Repo Gates, Check Changeset and Governed Surface Queue Guard are green. |
Disposition:
open_questions:- The stored-row second read: A, leave it, as record ③ answers it. That read is the version head through
SysMetadataRepository.headAt, the one write-address resolution whose contract forbids a second derivation. A stored-row object is at 2 reads, down from 5, and the PR discloses it. The card's wanted is "one read of an object's served overlay row", which holds for every object, and its measured case (the console open's code-defined objects) is at 1. SoFixes #24371closes the card. If a stored-object measure ever pulls for threading the head through the one read, that is its own card. - The single read's shape: A, one
$inquery. Two parallel reads are two statements on one connection, and a memo would outlive the call.
- The stored-row second read: A, leave it, as record ③ answers it. That read is the version head through
deviations: all accepted, as record ③ judges them: the test-only files outside the claim (forced by the$inshape; each double gained a$inarm and nothing looser), the ledger rows, Postgres measured from/tmp, the per-commit pushes and the footer form.- Record F1 (case-exact selection on a case-insensitive SQL collation): an acceptance note, now in the PR body. It moves only names the save doors already refuse, toward the case-exact contract every other layer keeps. Not a card.
out_of_scope_findings: the three loose doubles are fixed in this PR. Nothing to file.
Landing: ordinary queue (pr_ready + automerge_enable through the relay). Every check on this head is green or a rostered skip. The seat's body edit re-runs the body-triggered guards. Fixes #24371 then closes the card. #24360 (the engine copy strings, waiting on protocol.ts) can be claimed after this lands.
Generated by Claude Code
Fixes #24371
Clause-②: no (no accept set or public surface moves: the same answer with fewer reads; owes the seat's review of record at
TIER_DEFAULTbefore the queue)What changed
Every
GET /api/v1/data/OBJECTgoes through the API-exposure gate, which reads the object by name (getMetaItem). That read sent three active-row lookups to the store one at a time, five round trips in all:packagelessAddress), one read with no limit;findServedOverlayRow), onefindOneper spelling (object, thenobjects);overlayLockLayerAt), one read per spelling with no limit.Now
readMetaItemreads the item's ACTIVE rows once, for both spellings (readItemActiveRows:type IN (canonical, other), name, state = 'active', which uses the(type, name, …)overlay index). Each lookup then picks its rows from that read using the samewhereit used to send the store (rowMatchesOverlayWhere). Theversiontoken reuses the package binding the read already selected, so it no longer repeats lookup 1. The rows are read once per call and never kept, so the next request sees a metadata write.Only
packages/metadata-protocol/src/protocol.tschanges at runtime. The other callers of the three lookups (getMetaItemLayered, the_lockgate, the save and delete doors) are unchanged: each still reads the store itself.Trace (M1): measured, before any fix
A real backend (
pnpm dev:todo, SQLite, knex query log) and one signed-inGET /api/v1/data/todo_taskon base6a49efb9logged exactly the five statements the card names, in this order. All five come from the singlegetMetaItemcall thatenforceApiAccessmakes (packages/rest/src/rest-server.ts):findDataitself made nosys_metadataread: the registry already holds the definition. No othersys_metadataread happens per request. Thesys_metadata_activationread in the session prelude is a different table and belongs to #10757.H1 was confirmed, with one correction. The fifth read is not a second caller. It is
packagelessAddress, the package-less binding the served-row lookup is given.After the fix, the same request logs one statement:
Latency (M2): measured at 50 ms RTT
I ran Postgres 16 locally behind a TCP delay proxy that adds 25 ms each way. The todo app ran on a database seeded fresh for each leg. Each leg did 3 warm-up reads, then 15 timed
GET /api/v1/data/todo_taskas the signed-in admin. The before leg ranprotocol.tsfrom base6a49efb9; the after leg ran this branch.sys_metadatastatements per requestThe 211 ms saved is about four round trips. With ids and timestamps masked, the response bodies of the two legs are identical (8 records). These are wall times on a shared box.
Pins
Count, at the door.
packages/rest/src/rest-data-read-one-overlay-read.test.tsuses the realRestServer, the real protocol, the realSchemaRegistryand the realfindData, over an engine double that records everysys_metadataread:type: { $in: ['object', 'objects'] };version;enablewhitelist, stored under either spelling, still refuses a list 405;Answers, differential.
packages/metadata-protocol/src/protocol.item-active-rows-one-read.test.tscompares each lookup made from the one read with the store's own answer. It covers 76 arrangements ofobjectrows and 584 of a bundled type (email_template): both spellings, the package-unbound row and two packages, canonical and non-canonical member, three lock states, two store orders. Every address is checked. Each lookup returns the same answer either way.Ablation (one, on the behaviour the pins test). I swapped
activeRows = await this.readItemActiveRows(…)for an undefined read throughscripts/ablation-replace.mjs. On disk the anchor count went 1 → 0 and the marker count 0 → 1.ablation-dist-preflightthen reported the marker present in 2 built files. The count rows went red (5 failed, 39 passed across this file andrest-exposure-gate-one-object-read.test.ts):Every answer row stayed green. Restore leg:
git hash-objectmatches the HEAD blob,--absentpreflight passed with a clean tree, and the same run gave 44 passed.Pin sweep: test doubles that answer
whereby equality onlyThe single read sends
type: { $in: [...] }. Many hand-writtensys_metadatadoubles compared everywherevalue by plain equality, so the$invalue matched nothing and the stored row silently vanished. That made 27 metadata-protocol files fail (5803 tests), plus 8 in rest, 11 in objectql and 1 in runtime.Each double now compares
$inmembership as a driver does. It still refuses (throws on) every other operator it refused before. Every real driver (sql, memory, mongodb) already supports$in, andfindEnvironmentRowsalready sends$nin.packages/objectql/src/protocol-meta.test.tshad pins on the old read sequence: thefindOneorder['app', 'apps', 'app'], and overlay rows served throughfindOnemocks. These were flipped to the new contract: onefindwith$in, then the one headfindOne. The flipped rows carry weight; they are the ones the ablation would turn red.Three doubles (
protocol.save-flow-canonicalization,protocol.runtime-authoring-gate, objectqlprotocol-publish-rollback) answeredfindwithout filtering byname. They now filter byname, because the served row now comes fromfind.Verification record (final head
f95842334, merged withorigin/mainat5da2e888)Build:
pnpm build(76 of 76 tasks), then the touched closure.Typecheck, exit 0 with 0 errors for each of
@objectstack/metadata-protocol,@objectstack/rest,@objectstack/objectqland@objectstack/runtime. The rest and metadata-protocol test layers compile; for rest,check:test-typecheckpassed with 0 debt.Tests (exit 0):
--project unitThese are every package whose tests construct the protocol over a double. The two rest files and one objectql file that the merge brought in were re-run on the merged tree (21 and 107 passed).
Gates:
node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstackwith no paths derived 79 commands onf95842334. All 79 were run with the exit code captured before any pipe, and all exited 0.--ranreports:79 derived, 79 run, 0 NOT-MEASURED, 0 UNRUN(a derived zero; every line carries its exit code).check:engine-double-contractasked for the two new doubles to be recorded, andscripts/engine-double-contract.pinned.jsongained two rows from--write.Not run locally:
test:repoand the repo-wide lint, which CI owns. Thecliintegration tier was not run, as declared for CI: no spawn entry was touched.Files outside
domain:engineThe runtime change stays in
packages/metadata-protocol/src/protocol.ts. The sweep touches test files only, inpackages/rest/src(9, one of them the new count pin),packages/objectql/src(11) andpackages/runtime/src/domains(1), plus the ledgerscripts/engine-double-contract.pinned.json.Acceptance notes
sys_metadataread per data read. An object with no stored row (every code-defined object, which covers all of the console's opening reads that the card measured) meets it: 5 → 1. An object with a stored environment row still makes a second read: the head read behindgetMetaItem'sversion. It goes throughSysMetadataRepository.headAt, the repository's one write-address resolution, whose own contract forbids re-deriving it elsewhere. Such an object is now at 2 (measured live: 3 before theversionchange, 2 in the count pin), down from 5 by the trace on base. The base count for a stored-row object was traced, not measured live.getMetaItemLayered, the_lockgate on writes, the save and delete doors._ci), a by-name read whosename,?package=or bundle member differs from the stored row only by letter case used to be served that row and its lock. It is now not served it. Every grammatical address is unaffected, because the save and publish doors refuse a name outsideMETADATA_ITEM_NAME_PATTERN(lowercase snake_case). The registry, the metadata service and the/dataregistry gate were already case-exact, as SQLite and Postgres always were.Generated by Claude Code