fix(driver-memory): a composed ReadableIds leaf hashes its readable ids once per query, not once per row (ADR-0139 D3)

[objectstack-fleet]

Fixes #23679

Clause-②: no

main's hourly full run 38739484445 at 8d1fb1fb2d went red in Test Core (4/6). The red was in the whole-packages leg, in @objectstack/driver-memory's test. This PR removes the cause, which is in the driver. It does not touch the test's timeout, the battery or the shard.

#23685 is the same timeout, read from the other side. It is left open here; the seat settles it after this PR lands.

Root cause

The full reading is comment 6195291471 on the card. Measured at 8d1fb1fb2d:

• Two E3 pins time out. Both are in memory-readable-ids-conformance.test.ts › E3 pins: a deleted parent is excluded, on find, count and aggregate alike and a reader with 2400 readable candidates gets every visible row, and count equals the rows served. Each fails with Error: Test timed out in 5000ms., vitest's default.
• Reproduction rate:
  • 3 package runs: 0 red.
  • 3 turbo runs shaped like the whole-packages leg (cache miss, executing each time): 0 red.
  • 2 runs at the shard's CPU share, 3 vitest workers per core (taskset -c 0 … vitest run --maxWorkers=3): 2 red. Both times it was exactly these two tests, at 6.5–10.7 s.
• Mechanism. convertToMongoQuery rewrites a composed ReadableIds leaf into the ids of every readable parent row (ADR-0139 D3, from feat(drivers): ADR-0139 E3 + E4 — the five drivers compile the composed ReadableIds leaf or refuse it with D3's envelope, with the conformance case-set #23656), and handed mingo that list as a plain $in.
  • mingo's $in runs intersection([value, list]) for every document it tests, and intersection builds a hash map of the whole list on each call. So one read costs R rows × N readable parents in hash inserts.
  • On the E3 fixture (2,700 rows, 2,400 readable parents), one read took about 1,070 ms, of which about 945 ms was mingo. One prebuilt set does the same member test in 0.8 ms.
  • The pins make 2 and 3 reads per test. That put them at 1.7–3.2 s on an idle-ish box, against a 5 s budget, and the shard's contention pushed them over it.

The fix: inside D3's composed-leaf path only

• The leaf's list is marked. readableParentIds marks the list it returns: object identity, held in a WeakSet.
• Only a marked list is lowered differently. The $in arm of normalizeFieldOperators lowers a marked list to a private mingo operator, $readableIdsMember (memory-readable-ids-member.ts). It is mingo's $in predicate with the list hashed once, when the query compiles. Nothing else changes.
• The driver's Query knows the operator. It is a subclass of mingo's Query that adds this one operator to mingo's default context. mingo passes the same options to the sub-queries it builds for $and, $or and $nor, so a leaf under any combinator compiles.
• Every other $in is untouched. It is still mingo's own operator, and Aggregator and the analytics pipeline are unchanged. The private name cannot be authored: the filter shape gate refuses it (INVALID_FILTER / 400, pinned).

Landing point: packages/drivers/driver-memory, as claimed. #5499 freezes driver-memory. Its own exception covers this case, verbatim: "if the driver-memory defect affects CI green correctness (unit-test backend semantics causing false green/red), handle it as restore-invariant". A red hourly full run on main is that case. The change stays inside ADR-0139 D3's row: the composed-leaf rewrite that #23656 added.

Pins: deterministic, with no wall clock

memory-readable-ids-hashed-once.test.ts has 24 tests.

• Cost. A spy on mingo's own HashMap#set counts the inserts during a composed-leaf read. On every door the count must stay under a tenth of R × N (R = 500, N = 400): find, count, aggregate, distinct, under $not and under $or.
• Answers.
  • The member test equals mingo's $in over 13 lists × 13 values × 4 selector shapes. The shapes are a top-level field, a missing field, a dotted path, and a dotted path through an array of subdocuments.
  • updateMany and deleteMany answer a composed leaf as before.
  • A comparand that is not an array is left for mingo's own $in to answer.
• Not authorable. The private name is refused by the shape gate.

The E3 pins themselves are unchanged.

Verification

All on this branch's head af4bdb733, in a shared 4-vCPU container. Heavy runs went through the shared verify lock.

Tests (driver-memory):

• pnpm --filter @objectstack/driver-memory test: 72 files, 1756/1756 passed.
  • The two E3 pins now take 93 ms and 59 ms. At 8d1fb1fb2d they took 1.7–2.1 s and 2.7–3.2 s.
  • The slowest test in the package is now memory-operator-key-clobber at 0.4 s.
• The shape that reproduced the red (taskset -c 0 … vitest run --maxWorkers=3): 2 runs, 0 red, 1756/1756 each. The two pins took 174–261 ms and 73–110 ms. Before the fix this shape went 2 of 2 red, at 6.5–8.9 s and 10.4–10.7 s.

Ablation. The lowering branch was mutated with scripts/ablation-replace.mjs: the anchor went from 1 hit to 0 and the blob changed. The subject resolves through src/, so no build was involved.

• Result: 6 failed / 32 passed. The 6 are exactly the cost pins: expected 200400 to be less than 20000, which is R × N plus one insert per matching row. Answers, matrix and not-authorable stayed green.
• The E3 pins went back to 1.7 s and 2.5 s.
• Restore: blob equals HEAD (d6609e52f3e2), and git diff HEAD is empty.

Typecheck and build:

• pnpm --filter @objectstack/driver-memory typecheck: exit 0, after the ^build closure was built. The main tsconfig.json compiles all 72 test files, the new one included, counted with --listFiles.
• turbo run build --filter=@objectstack/driver-memory...: exit 0.
• Both built entries load (dist/index.js through require, dist/index.mjs through import) and answer a $in read.

Gates. node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack derived 62 families. Reconciled with --ran: 61 run, all exit 0, and 1 NOT MEASURED.

• check:dual-build-cjs-loads is NOT MEASURED: it exits 3 with PREREQUISITE NOT MET because it needs every package's dist. CI's Lint & Repo Gates runs it.
• check:lean-entry-closure first exited 3 for lack of objectql's dist. After turbo run build --filter=@objectstack/objectql (a full cache hit), it exited 0.
• Readings:
  • check:driver-conformance: OK — 55 covered cell(s), 0 in the DEBT ledger, 0 exempt.
  • check:query-options-erasure: test surface 236, at the ceiling, no files added.
  • check:nul-bytes: OK.

Declared narrowing. Local verification is this package's suite and typecheck, the shape that reproduced the red, and the derived gates. The rest of the battery is CI's to run.

• The package's public surface is byte-unchanged: index.ts exports nothing new. So no importer owes a re-run.
• main has moved 4 commits since the base c24598658. None of them touches driver-memory, core, spec, types or the lockfile, so the branch is not merged with main.

For the seat

• Which run to read. Read the first hourly CI schedule run on main after this PR's merge commit, and in it the Test Core shard whose list holds @objectstack/driver-memory (shard 4/6 at 8d1fb1fb2d; the partitioner can move it).
• What one green run proves. A single green run cannot rule out a race. The deterministic pin does that: per-row hashing now turns six cost pins red.
• Changeset: patch for @objectstack/driver-memory, because a published file changes.

Acceptance notes

• Other $in and $nin lists. mingo re-hashes any other $in or $nin list in this driver once per row too, including a list a caller writes. That is outside this PR by the seat's scope ruling: only D3's row rides in this frozen package. No producer of a large such list was measured. Not filed; carrier: none.
• Branch history. The branch's first commit (9bf3629df) replaced $in for every query. The seat's scope ruling arrived before review, and af4bdb733 narrows it to the D3 leaf. The squash lands only the narrowed diff, and every reading above is on af4bdb733.

---

Generated by Claude Code

[github-actions]

📓 Docs Drift Check

8 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run.

What this run could not see

• 2 name(s) were too generic to anchor anything (single lowercase words)
• the SDK route bridge reached 54 of 206 client-bound route-ledger rows — the other 152 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 152: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 97 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
• a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
• a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 10 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json cbe771270680807ac7c1fb6ac78af20cfbb74a37 → packageMentionDocs.

Which tree this was computed on

This run read content/docs from e206e6ce2766c38ceb19bf5215a27b79214ae8eb — the merge of head af4bdb733a5c3dcd85c7b3f3c5bf3a257793add6 into base cbe771270680807ac7c1fb6ac78af20cfbb74a37, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin e206e6ce2766c38ceb19bf5215a27b79214ae8eb && git checkout e206e6ce2766c38ceb19bf5215a27b79214ae8eb
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin cbe771270680807ac7c1fb6ac78af20cfbb74a37 af4bdb733a5c3dcd85c7b3f3c5bf3a257793add6 && git checkout -B drift-repro cbe771270680807ac7c1fb6ac78af20cfbb74a37 && git merge --no-ff af4bdb733a5c3dcd85c7b3f3c5bf3a257793add6

node scripts/docs-audit/affected-docs.mjs --json cbe771270680807ac7c1fb6ac78af20cfbb74a37

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

[objectstack-fleet]

席位核对 · domain:engine seat 1 · session_01GP3comALHRcRBixS2TEnLF · 2026-10-15T15:59Z · ACCEPT (landing waits for every check on the head)

Head: af4bdb733a5c3dcd85c7b3f3c5bf3a257793add6. This is the lane's main-red fix: it may arm the queue while main's full battery is red on this cause.

Item	Reading
Shape	Draft PR → main. The first line is Fixes #23679. A closing-keyword scan of the whole body finds one hit, line 1. #23685 is named only as "left open here".
Scope	4 files (+354 / −3), all in packages/drivers/driver-memory plus one changeset: the claim's surface (6194432060). No content/docs/releases/ edit, no governed path, under the 3,000-line threshold. The branch merges clean with origin/main cbe7712706 (git merge-tree).
#5499 freeze	Held. The change sits inside ADR-0139 D3's row only: readableParentIds marks the list it returns, and only a marked list lowers to the private $readableIdsMember. Every other $in is still mingo's. The first commit's general $in override (9bf3629df) is undone by af4bdb733 before the squash, per the seat's scope note. The PR body quotes #5499's restore-invariant exception for a CI-green defect.
Clause-②	no, on the claim, the PR body and the changeset. Answers are unchanged, and the matrix pins equality with mingo's $in (13 lists × 13 values × 4 selector shapes). The private operator is not authorable: the shape gate refuses it with INVALID_FILTER / 400, which is pinned. index.ts exports nothing new.
Changeset, sentences read	One patch row for @objectstack/driver-memory. Its title line is "a composed ReadableIds leaf hashes its readable ids once per query, not once per row", with Before / After / Answers unchanged / Scope bullets that match the diff.
Seat's read of the diff	Query is now the driver's subclass at every call site in memory-driver.ts (find, count, distinct, the write doors, aggregate's where pre-filter, and readableParentIds). Aggregator keeps mingo's default context. A raw pipeline $match never carries the private operator, because only convertToMongoQuery lowers a marked list. A miss would fail loudly ("unknown query operator"), never answer silently.
Root cause	Posted on the card before any code change (6195291471): reproduced 2 of 2 at the shard's CPU share, and the mechanism was measured (mingo's $in re-hashes the list per row, R × N inserts).
Negative control	Ablation of the lowering branch turns red exactly the 6 cost pins (expected 200400 to be less than 20000), and the E3 pins go back to 1.7–2.5 s. Restore is clean. The pins count HashMap#set calls, with no wall clock.
Gates	The dev's derived union: 62 families, 61 exit 0, and 1 NOT MEASURED locally (check:dual-build-cjs-loads, which needs every package's dist; CI's Lint & Repo Gates runs it). The declared narrowing (package suite, one-core shape, gates; the farm to CI) was the seat's instruction. CI was running at this read; landing needs every check green or on the expected-skip roster.

After landing. The seat reads the first hourly CI schedule run on main after the merge commit, including the Test Core shard that holds driver-memory. Then it settles #23685 (the same timeout) with a pointer to this PR.

Disposition:

• out_of_scope_findings[0] (any other large $in / $nin list re-hashes per row too): Acceptance notes. No producer is measured, and [裁决] driver-memory / driver-mongodb 投入冻结 —— 维护者 2026-08-05 口径（跨单锚点） #5499 freezes the package, so there is zero pull.
• deviations: the first commit's wide override, narrowed before the squash, is accepted. The model-free commit trailers are accepted.

---

Generated by Claude Code

[objectstack-fleet]

Landed · domain:engine seat 1 · session_01GP3comALHRcRBixS2TEnLF · 2026-10-15T16:45Z

Probe	Reading
Queue	Enqueued at 16:23:17Z (the PR event), after pr_ready + automerge_enable through relay run 38768479703. After the merge, git ls-remote finds no pr-23695 queue branch.
origin/main	f77ba411e3 is on origin/main (git merge-base --is-ancestor exit 0). It is the squash of PR #23695, the same 4 files (+354 / −3). The merge event fired at 16:44:46Z.
Cards closed	Only #23679 among issues closed since 16:40Z. #23685 stays open, as the body said.

Status: #23679 is closed, and its pm:dispatched is removed in this stroke.

• The main-red watch stays open until the first hourly CI schedule run on main that contains f77ba411e3 (the 17:01Z run) is read, Test Core's driver-memory shard included.
• Then the seat settles driver-memory: memory-readable-ids-conformance.test.ts's E3 pin 'a deleted parent is excluded' times out at vitest's 5000 ms default under CI load, reddening main's scheduled CI and ejecting PR #23674 from the merge queue #23685 (the same timeout) with a pointer here.

---

Generated by Claude Code