Skip to content

Add the runtime boundary ratchet (script, baseline, CI job) - #15135

Merged
cderv merged 17 commits into
mainfrom
rt/02-runtime-boundary-check
Oct 21, 2026
Merged

cderv merged 17 commits into
mainfrom
rt/02-runtime-boundary-check

Conversation

@cderv

@cderv cderv commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Description

PR 02 of the runtime migration (plan): a ratchet that counts Deno.* calls and direct std imports in five scopes (src, src-js, resources, package/src, tests), a committed baseline, and a Runtime Boundary CI job that fails if any count exceeds it. Also tests/asserts.ts as the single assertion import for tests.

package/dist/bin/quarto.cmd run --dev tools/check-runtime-boundary.ts on this branch (Windows 11, exit 0):

Runtime boundary check (baseline: tools/runtime-boundary-baseline.json)
scope         Deno.* calls            std-import files
src           994 (baseline 994)      49 (baseline 49)
src-js        7 (baseline 7)          0 (baseline 0)
resources     10 (baseline 10)        1 (baseline 1)
package/src   240 (baseline 240)      3 (baseline 3)
tests         826 (baseline 826)      177 (baseline 177)
OK: no count exceeds the baseline.

Baseline: src 994/49, src-js 7/0, resources 10/1, package/src 240/3, tests 826/177 — i.e. main's 825/177 plus the unit test's one Deno. occurrence (Deno.writeTextFileSync in writeFixture). --accept-tests-increase was not used.

New test files and their Deno. counts:

  • tests/unit/tools/check-runtime-boundary.test.ts: +1
  • tests/unit/asserts.test.ts: 0

Notes for reviewers:

  • 869487706 + 3cc755654 together change exactly one README cell: the first picked up a format-on-save reflow of dev-docs/runtime-migration/README.md, the second reverts it. Net diff of the pair is the PR 02 status cell (planned -> in progress).
  • 31d4c9bdd rewords two test expectation strings (built from DENO instead of literal Deno.*) and lowers the tests baseline 828 -> 826 in the same change.
  • The branch was rebased onto current main before opening; the two tip commits (cb4e500c8, efcada396) are docs-only.

Checklist

I have (if applicable):

  • referenced the GitHub issue this PR closes
  • updated the appropriate changelog in the PR
  • ensured the present test suite passes
  • added new tests
  • created a separate documentation PR in Quarto's website repo and linked it to this PR
AI-assisted PR
  • AI tool used: Claude (Sonnet) via Posit Assistant for the implementation sessions, run per-session against the plan's task steps; Kimi (Moonshot AI) via Posit Assistant for the final session (push, PR, docs log)
  • Codebase grounding: local clone
  • Human review: I have reviewed, tested, and verified the AI-generated content before submitting.

Note: autonomous AI agents submitting PRs without human oversight are not permitted — see the Code of Conduct.

cderv added 16 commits October 21, 2026 16:05
PR 02 Task 1 failed as written: re-exported from "testing/asserts", the
seam's AssertionError is not the class its assert() and fail() throw.
testing/asserts resolves to @std/testing@0.224.0/asserts.ts, a deprecated
shim that delegates to @std/assert but declares its own AssertionError.
@std/testing 0.225.3, the last 0.x, has the same design and 1.0 removed
the module, so no version of it satisfies the test.

Decided 2026-10-21: the seam re-exports from "@std/assert" through a new
import-map key pinned at 0.224.0, the version the shim already uses. A raw
jsr: specifier would also pass on Deno, but PR 26 makes package.json
dependencies an exact mirror of src/import_map.json, so it would leave
@jsr/std__assert undeclared on Node until PR 31. With the key, PR 31
only bumps its version. @std/assert joins kStdModules so a direct import
outside the seam is still counted. The evidence is in PR 02's "Decisions
needed".

Downstream: _TEMPLATE.md names the seam's source, PR 22's table gains
the key, PR 23 keeps the specifier when it extends the seam, PR 31 Task 1
bumps the existing key instead of adding it.

Two gaps found on the way:
- The contract tests of PRs 08, 09 and 14 imported testing/asserts, which
  the tests ratchet counts, while those plans promise an unchanged or
  lower baseline. They now import tests/asserts.ts.
- Eight src files import assert from testing/asserts and no plan owns
  them, so PR 25's zero would fail on them. Recorded as an open decision
  in PR 22 with a recommendation.

PR 02 also carries the executor's execution log, to be removed before
the PR merges.
Counts Deno. occurrences and direct std imports (import-map std names,
deno.land/std URLs, jsr:@std specifiers) per scope: src, src-js,
resources, package/src, tests. deno.land/x packages are not std.
Checkbox ticks for completed steps and the session-4 execution log
entry: stopped at Task 3 step 4 (the probe target src/core/text.ts was
already a counted std-import file at b86f98d); fresh baseline numbers
recorded (src 994/49, src-js 7/0, resources 10/1, package/src 240/3,
tests 828/177).
The ratchet's std-import probe appended a deno.land/std URL import to
src/core/text.ts and expected src std-import files to rise from 49 to 50.
text.ts already imports fmt/colors, so it is one of the counted files and
the count cannot rise; the expected error never appears. It was wrong when
the plan was written, not drift on main.

The probe now targets src/core/mime.ts, which imports no std module. Run
on Windows against the measured baseline, the amended step gives every
expected error and exit 1.
hasStdImport matched only the first import on each line, so an allow-listed import (`async`) placed before a counted one (`fmt/colors`) hid it, and a `from` separated from its specifier by a newline was missed. It now scans every import in the comment-stripped text.

A baseline scope missing one of its two counts made the comparison against the missing value always false, silently disabling that constraint. parseBaseline now rejects any scope whose denoCalls or stdImportFiles is not a non-negative integer, and the script exits 2 with a message instead of comparing.

The repository counts are unchanged by the stricter scan, so the committed baseline stays valid.
Step 5 diffed and restored the baseline before step 6 committed it.
On an untracked file git diff prints nothing whatever happened, and
git checkout -- aborts on the pathspec and leaves the probe lines in
place. Step 5 now stages the baseline first.

Also carries the measured counts into step 5 and the step 6 commit
message, the 13-test count after the parseBaseline tests, and how to
warm the launcher's deno_cache for the step 4 URL probe.
Measured on main at 502f284: src 994 Deno. occurrences / 49
std-import files (allow-listed import-map seams not counted),
src-js 7 / 0, resources 10 / 1, package/src 240 / 3, tests 828 / 177.
The previous commit picked up a format-on-save markdown reflow of
README.md alongside the intended PR 02 status cell change. This
restores the file's original formatting; the net diff of the two
commits is the status cell only.
…the unit test once

The two expectation strings contained a literal "Deno.* calls", so the
ratchet's own regex matched the unit test file three times, not once as
designed (quarto-cli-81ld.51). Building them from the DENO constant like
the fixtures keeps the file's text invisible to the ratchet; the tests
baseline drops from 828 to 826 Deno. occurrences.
The plan is public documentation; the decision and its rationale stay,
the private issue-tracker identifier does not.
Same yamark settings as the rest of dev-docs/runtime-migration
(sentence wrap, preserved table widths). Applied to the docs this
branch touches that main's formatting pass did not cover.

The YAML blocks embedded in the plans are left as written: yamark
rewrote them (stripped quotes, folded long `run:` lines into `>-`),
and the plans show those files verbatim.
@posit-snyk-bot

posit-snyk-bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
✅ Open Source Security 0 0 0 0 0 issues
✅ Licenses 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@cderv
cderv merged commit cf768e1 into main Oct 21, 2026
52 checks passed
@cderv
cderv deleted the rt/02-runtime-boundary-check branch October 21, 2026 15:00
cderv added a commit that referenced this pull request Oct 22, 2026
Pull request runs now compare against the base branch, so a PR is judged
on its own change. Comparing absolute counts blamed every open PR for an
increase merged to main by another PR (#15136 after #15134). The check
nets increases out per Deno API and per std module before any file is
blamed, so a renamed file or a moved call is not a finding.

An increase is an error only for a Deno API that src/deno_ral covers
equivalently (build.os and build.arch, in scopes that can import it).
Every other increase is a warning on the added line, and the job
summary gives the per-scope delta and the baseline bump a maintainer
accepts with --accept-increase. Std imports get only a suggested
replacement, since a name check cannot show that a wrapper exports what
the file uses. Pushes to main and manual runs keep the absolute baseline
gate.

- Delta mode reads the baseline committed at the base revision. A PR
  that adds nothing while main is above baseline gets only a notice.
- Delta mode lists the whole base tree rather than passing every changed
  path to git, which keeps long comparisons under the Windows
  command-line limit.
- Counts use a prototype-free dictionary, so an identifier such as
  constructor is an ordinary key.
- src/core/lib is browser-shared and cannot import src/deno_ral, so an
  increase there is a warning, not an error.

Related to #15134, #15135 and #15136.
cderv added a commit that referenced this pull request Oct 22, 2026
The plans themselves merged to main in #15133, so HANDOFF.md no longer
points readers at the fork branch they were drafted on, and "How to
resume" starts from main.

PR 02 merged in #15135 and PR 02c in #15153. The README sequence row
for 02 still said "in progress", and HANDOFF.md still named PR 02 as
the first plan to open. Plans check their dependencies against that
table at step 0, so it now says merged, and the handoff names 00a
then 00b as the next plans in README order.

docs: runtime migration, wrap-up checklist for agents

The README already says who moves a row in "The sequence" and when,
but a merge on GitHub runs no session, so the status column and
HANDOFF.md drifted: PR 02 still read "in progress" after it merged.
Findings made while executing one plan that change another plan's
prerequisites were also left only in PR threads.

An AGENTS.md in this directory is read by any agent working on the
plans, the implementer included, so the reminder sits where the work
happens. Findings go to README "Open decisions with an owner" rather
than into the affected plan, which its step 0 re-reads anyway. The
file is removed with the directory when the migration ends.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants