Skip to content

feat(agents): the runtime seam for every catalogue language, then plotnine and altair (plan steps 1a and 1b) - #12404

Merged
MarkusNeusinger merged 7 commits into
mainfrom
claude/vigilant-mccarthy-8euyyw
Oct 25, 2026
Merged

MarkusNeusinger merged 7 commits into
mainfrom
claude/vigilant-mccarthy-8euyyw

Conversation

@MarkusNeusinger

@MarkusNeusinger MarkusNeusinger commented Oct 24, 2026 •

Copy link
Copy Markdown
Owner

Summary

Two commits, one per plan step; each reviews on its own (agentic/specs/261024-agent-all-libraries.md). Say so if you want them as two PRs.

  • Step 1a (commit 7fc230e), the runtime seam, no behaviour change for matplotlib and seaborn: the renderer holds one fixed render command per language in agents/renderer/commands.py, keyed by the job's (language, library) and never taken from the request; the probe harness gains an exec mode (harness.py --exec -- <command>) that runs a non-Python command under the same limits, start and end lines, launcher retry and kill path; GET /status reports the toolchains the image carries and a job in a language without one answers 503 runtime_unavailable. On the agents side the registry agents/anyplot/render/runtimes/runtime_for hands the pipeline, the theme toggle, the local backend and the eval harness one RuntimeAdapter per library, which also carries the language's code conventions, error grammar, export and the render time a repair reserves. The exported code's artifact name follows the language (plot.py, plot.R, plot.jl, plot.js, plot.tsx) through the wire, the BFF, the feedback case and the result card.
  • Step 1b (commit 63d98e0), plotnine and altair through the Python runtime: one save rule per library (regions.SAVE_RULES) replaces the savefig rule in the regions, the validator, the normaliser, the readiness scan and the edit applier (plotnine's save and ggsave, altair's save with its second plot-<theme>.html and the PIL padding block). The validator gains both import profiles, the import surveys' bans and the routes their adversarial pass found (altair's image mark and from_dict, the narwhals frame behind limit_rows, transformed_data, plotnine's .environment), parses plotnine's aesthetic and facet strings as code and takes them only as literals it can read, and refuses formula= and from_formula. The harness admits the HTML write, hands plotnine's Environment.capture the plot's frame, lets Environment.eval run the walked strings, and attaches an Agg canvas so the probe measures plotnine's figure. The altair command row runs without RLIMIT_AS (vl-convert's V8 aborts under 64 GiB of address space). The renderer image installs both libraries and links DejaVu Serif and Sans Mono for vl-convert; ci-image.yml renders one plot of each through the harness. The eval fixtures gain 120 cases and two smoke cases.
  • Commit b182dd4, the two blockers CI raised on 63d98e0, neither in the steps' own code: CodeQL's py/polynomial-redos (high) on policy._FENCE_TAG, whose two blank runs around an optional slash backtracked quadratically on user text; the blanks after the slash now sit inside the slash's own group, the matches are the same, and the CodeQL CLI (2.27.3, the workflow's suite and paths-ignore) no longer reports it locally. And the memory-floor executor test, which killed the run 60 ms after the launch and then compared the deletes with a launch log the loaded runner's fake launcher had not written yet; the floor is now crossed once the launch is in the log.
  • Commit f4ceff0, five in-process tests for the seam's lines the suite reached only through a sandbox child: the local factory's runtime resolver, opening.assess for a library without a runtime, frame_for_readers, _renderer, and harness.main --exec with set_limits patched out (the exit codes, the start and end lines, the usage error). Patch coverage 96.4 %; what stays uncovered is the nine Protocol stubs in contract.py and four harness lines only the sandbox child runs.
  • The phase-2 plan records the spike measurements, the Fable second opinion, the 1b decisions that settle its text, the eligibility sweep and the owner questions with their defaults.

Nothing changes for users until AGENT_LIBRARIES names plotnine or altair (the default stays matplotlib,seaborn; the plan's 80 % matrix gate decides).

Eligibility (agents.evals.eligibility, 2026-10-25)

Library Files Eligible
matplotlib 325 285 (87.7 %), unchanged against main
seaborn 325 287 (88.3 %), unchanged against main
plotnine 321 264 (82.2 %)
altair 325 269 (82.8 %)

The plan lists every blocked file's cause. A harness sweep with the test venv rendered 258 of the eligible plotnine files and 258 of the eligible altair files.

Plan

agentic/specs/261024-agent-all-libraries.md

Test plan

  • uv run ruff check agents api core tests, ruff format --check, uv run mypy api core agents, uv run python -m tools.changelog check --base origin/main pass
  • uv run pytest tests/unit/agents tests/unit/api tests/unit/core: 11,798 passed (the tests/unit/workflows and agentic/regen failures of the cloud container are environmental, as noted for 1a)
  • New tests: the command table and its rows, the exec mode, the registry, the artifact names and their five copies, the save rules (regions, normaliser, readiness, edits), the plotnine and altair validator profiles with the reviewed submodule pins, the harness under the guard with plotnine and altair (HTML allowlist, frame reader, probe), the fixture generator
  • Local Docker: the renderer image builds with both extras (1.13 GB, +107 MB); the three CI render steps, the convert smoke and the multi-script render pass against it; measured in the image: plotnine 3200x1800 with a probe (max_rss 196 MB, cpu 1.9 s), altair 3200x1800 plus the HTML (max_rss 309 MB, cpu 2.3 s)
  • cd app && yarn lint && yarn fm:check && yarn type-check pass (1a); 1b changes no frontend file
  • The fence tests and the executor bound tests pass five times under -n 4 load after b182dd4
  • CI green on f4ceff0, the renderer-image job, CodeQL and the backend patch coverage included; the six Copilot threads of the first round are resolved
  • Owner: the matrix run for plotnine and altair against the deployed 1b renderer (AGENT_LIBRARIES=matplotlib,seaborn,plotnine,altair), then _LIBRARIES in agents/cloudbuild.yaml; the plan's open questions, or the defaults stand

The plan at agentic/specs/261024-agent-all-libraries.md takes "Use with my
data" from matplotlib and seaborn to the other 13 catalogue libraries: a
runtime registry, one fixed render command per language in the renderer, a
harness exec mode for Node, Rscript and julia, the toolchains in the renderer
image and a deterministic code profile per language, in five PRs. It records
the first spike measurements (the JS harness, the Python extras under the
probe harness, plotly and bokeh) and the owner questions with their defaults.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WbjVqBweiXjUZKp626ZFWm
Copilot AI lite review requested due to automatic review settings October 24, 2026 23:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The review identified unresolved inconsistencies in the plan’s scope, runtime contracts, validation commands, implementation architecture, and changelog summary.

6 open issues
What changed in this PR

This documentation-only PR adds a Phase 2 roadmap for expanding “Use with my data” from matplotlib and seaborn to all 15 catalogue libraries.

Changes:

  • Defines runtime registry, renderer, harness, toolchain, security, and evaluation plans.
  • Records spike results and open implementation decisions across Python, JavaScript, R, and Julia.
  • Adds a corresponding changelog fragment.
File Description
changelog.d/​agent-all-libraries.md Changelog entry describing the Phase 2 all-library expansion plan.
agentic/​specs/​261024-agent-all-libraries.md Detailed architecture, implementation steps, measurements, validation commands, and open decisions.

Review effort: Lite

Comment thread agentic/specs/261024-agent-all-libraries.md
Comment thread agentic/specs/261024-agent-all-libraries.md Outdated
Comment thread agentic/specs/261024-agent-all-libraries.md Outdated
Comment thread agentic/specs/261024-agent-all-libraries.md Outdated
Comment thread agentic/specs/261024-agent-all-libraries.md Outdated
Comment thread agentic/specs/261024-agent-all-libraries.md Outdated
…plan

The review split the foundation into the seam (1a) and one PR per library
pair, put R before JavaScript as the exec-mode pilot, moved bokeh and plotly
onto the JavaScript harness's Chromium as a second exec-mode child, added the
per-language error grammar and artifact name, the missed eval and test sites,
the RLIMIT_NPROC measurement (77 tasks for one Chromium render) and the Data
block coverage of the fixture specs as an owner task per language. The R
spike's numbers (1.03 GB image, 1.2 to 1.9 s renders, under the Python
limits) are recorded.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WbjVqBweiXjUZKp626ZFWm
The renderer holds one fixed render command per language in
agents/renderer/commands.py, keyed by the job's (language, library) and never
taken from the request; the probe harness gains an exec mode
(harness.py --exec -- <command>) that runs a non-Python command under the same
limits, start and end lines, launcher retry and kill path; GET /status reports
the toolchains the image carries, and a job in a language without one answers
503 runtime_unavailable. On the agents side the registry
agents/anyplot/render/runtimes/runtime_for hands the pipeline, the theme
toggle, the local backend and the eval harness one RuntimeAdapter per library,
which now also carries the language's code conventions, error grammar, export
and the render time a repair reserves. The exported code's artifact name
follows the language (plot.py, plot.R, plot.jl, plot.js, plot.tsx) through the
wire, the BFF, the feedback case and the result card. matplotlib and seaborn
behave as before; the table's other rows wait for their toolchains.

The plan records the Fable second opinion, the R and Julia spike numbers and
the Copilot review's corrections (status line, seven PRs, the matrix glob, no
kaleido path).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WbjVqBweiXjUZKp626ZFWm
@MarkusNeusinger MarkusNeusinger changed the title docs(agents): phase-2 plan for all 15 libraries in "Use with my data" feat(agents): runtime seam for every catalogue language, with the phase-2 plan Oct 25, 2026
@codecov

codecov Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.35974% with 17 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
agents/anyplot/render/contract.py 60.86% 9 Missing ⚠️
agents/anyplot/render/harness.py 90.24% 4 Missing ⚠️
agents/anyplot/code/validate.py 98.23% 3 Missing ⚠️
agents/anyplot/code/normalise.py 96.77% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

…ep 1b)

The runtime registry serves plotnine and altair next to matplotlib and
seaborn. One save rule per library (regions.SAVE_RULES) replaces the savefig
rule in the regions, the validator, the normaliser, the readiness scan and the
edit applier: plotnine's save and ggsave (the file is the second positional of
ggsave), altair's save with its second plot-<theme>.html file and the PIL
padding block that reopens the PNG; the protected final save is the last
module-level statement that writes the PNG. The validator gains the two
import profiles (plotnine and mizani on the matplotlib stack; altair and
PIL.Image), the surveys' bans and the routes their adversarial pass found
(altair's image mark and from_dict, the narwhals frame behind limit_rows,
transformed_data, plotnine's .environment), parses plotnine's aesthetic and
facet strings as code and takes them only as literals it can read, refuses
formula= and from_formula (patsy execs while it parses), np.save under every
rule by its receiver, and a save's format or path by position. The harness
admits the HTML write, hands plotnine's Environment.capture the plot's frame
through frame_for_readers, lets Environment.eval run the walked strings, and
attaches an Agg canvas to plotnine's figure so the probe measures it. The
renderer's altair row runs without RLIMIT_AS: vl-convert's V8 aborts under
64 GiB of address space. The renderer image installs both libraries and links
DejaVu Serif and Sans Mono for vl-convert; the CI job renders one plot of each
through the harness. The eval fixtures gain 120 cases and two smoke cases; the
matrix harness warns when AGENT_LIBRARIES does not enable a selected case's
library. Eligibility: plotnine 264 of 321, altair 269 of 325; matplotlib and
seaborn unchanged against main. AGENT_LIBRARIES keeps its default until the
owner's matrix run.

Also folds the low findings of the step-1a review: the docs name every code
artifact, the renderer's status row lists `languages`, a test pins the five
copies of the artifact name table, and the limits prose says which rows run
without RLIMIT_NPROC.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WbjVqBweiXjUZKp626ZFWm
@MarkusNeusinger MarkusNeusinger changed the title feat(agents): runtime seam for every catalogue language, with the phase-2 plan feat(agents): the runtime seam for every catalogue language, then plotnine and altair (plan steps 1a and 1b) Oct 25, 2026
… memory-floor test

CodeQL flagged `_FENCE_TAG` on this PR (py/polynomial-redos, high): two blank
runs around an optional slash are one ambiguous run, so `<` followed by a long
run of blanks backtracked quadratically on user text. The blanks after the slash
now sit inside the slash's own group; the matches are the same.

The memory-floor executor test killed the run after three watchdog samples
(60 ms), before a loaded machine's fake launcher had logged its `do`, and then
compared the deletes with an empty launch log (CI on 63d98e0, and locally
under -n 4). The floor is now crossed once the launch is in the log.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WbjVqBweiXjUZKp626ZFWm
Five tests reach the lines the suite covered only through a sandbox child:
`make_backend`'s resolver hands the local backend the registry's runtime with
the render timeout as its CPU limit, `opening.assess` keeps a library without
a runtime as it is and blocks it as unsupported, `frame_for_readers` answers
the caller's frame or None, `_renderer` attaches Agg's canvas to a bare
figure, and `harness.main --exec` frames a child, maps a signal death and a
missing command and refuses an empty command, with `set_limits` patched out
of the test process.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WbjVqBweiXjUZKp626ZFWm
@MarkusNeusinger
MarkusNeusinger merged commit 78ac0ce into main Oct 25, 2026
15 checks passed
@MarkusNeusinger
MarkusNeusinger deleted the claude/vigilant-mccarthy-8euyyw branch October 25, 2026 23:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants