Repository navigation
feat(agents): the runtime seam for every catalogue language, then plotnine and altair (plan steps 1a and 1b) - #12404
Merged
Conversation
The plan at agentic/specs/261024-agent-all-libraries.md takes "Use with my data" from matplotlib and seaborn to the other 13 catalogue libraries: a runtime registry, one fixed render command per language in the renderer, a harness exec mode for Node, Rscript and julia, the toolchains in the renderer image and a deterministic code profile per language, in five PRs. It records the first spike measurements (the JS harness, the Python extras under the probe harness, plotly and bokeh) and the owner questions with their defaults. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WbjVqBweiXjUZKp626ZFWm
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WbjVqBweiXjUZKp626ZFWm
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The review identified unresolved inconsistencies in the plan’s scope, runtime contracts, validation commands, implementation architecture, and changelog summary.
6 open issues
Add the required roadmap status line · New Resolve mismatch between five and six planned PRs · New Pass library identity for JavaScript rendering · New Clarify registry scope for implemented runtimes · New Remove rejected Kaleido architecture from the plan · New Make library selection valid for matrix cases · New
What changed in this PR
This documentation-only PR adds a Phase 2 roadmap for expanding “Use with my data” from matplotlib and seaborn to all 15 catalogue libraries.
Changes:
- Defines runtime registry, renderer, harness, toolchain, security, and evaluation plans.
- Records spike results and open implementation decisions across Python, JavaScript, R, and Julia.
- Adds a corresponding changelog fragment.
| File | Description |
|---|---|
changelog.d/agent-all-libraries.md |
Changelog entry describing the Phase 2 all-library expansion plan. |
agentic/specs/261024-agent-all-libraries.md |
Detailed architecture, implementation steps, measurements, validation commands, and open decisions. |
Review effort: Lite
…plan The review split the foundation into the seam (1a) and one PR per library pair, put R before JavaScript as the exec-mode pilot, moved bokeh and plotly onto the JavaScript harness's Chromium as a second exec-mode child, added the per-language error grammar and artifact name, the missed eval and test sites, the RLIMIT_NPROC measurement (77 tasks for one Chromium render) and the Data block coverage of the fixture specs as an owner task per language. The R spike's numbers (1.03 GB image, 1.2 to 1.9 s renders, under the Python limits) are recorded. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WbjVqBweiXjUZKp626ZFWm
The renderer holds one fixed render command per language in agents/renderer/commands.py, keyed by the job's (language, library) and never taken from the request; the probe harness gains an exec mode (harness.py --exec -- <command>) that runs a non-Python command under the same limits, start and end lines, launcher retry and kill path; GET /status reports the toolchains the image carries, and a job in a language without one answers 503 runtime_unavailable. On the agents side the registry agents/anyplot/render/runtimes/runtime_for hands the pipeline, the theme toggle, the local backend and the eval harness one RuntimeAdapter per library, which now also carries the language's code conventions, error grammar, export and the render time a repair reserves. The exported code's artifact name follows the language (plot.py, plot.R, plot.jl, plot.js, plot.tsx) through the wire, the BFF, the feedback case and the result card. matplotlib and seaborn behave as before; the table's other rows wait for their toolchains. The plan records the Fable second opinion, the R and Julia spike numbers and the Copilot review's corrections (status line, seven PRs, the matrix glob, no kaleido path). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WbjVqBweiXjUZKp626ZFWm
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
…ep 1b) The runtime registry serves plotnine and altair next to matplotlib and seaborn. One save rule per library (regions.SAVE_RULES) replaces the savefig rule in the regions, the validator, the normaliser, the readiness scan and the edit applier: plotnine's save and ggsave (the file is the second positional of ggsave), altair's save with its second plot-<theme>.html file and the PIL padding block that reopens the PNG; the protected final save is the last module-level statement that writes the PNG. The validator gains the two import profiles (plotnine and mizani on the matplotlib stack; altair and PIL.Image), the surveys' bans and the routes their adversarial pass found (altair's image mark and from_dict, the narwhals frame behind limit_rows, transformed_data, plotnine's .environment), parses plotnine's aesthetic and facet strings as code and takes them only as literals it can read, refuses formula= and from_formula (patsy execs while it parses), np.save under every rule by its receiver, and a save's format or path by position. The harness admits the HTML write, hands plotnine's Environment.capture the plot's frame through frame_for_readers, lets Environment.eval run the walked strings, and attaches an Agg canvas to plotnine's figure so the probe measures it. The renderer's altair row runs without RLIMIT_AS: vl-convert's V8 aborts under 64 GiB of address space. The renderer image installs both libraries and links DejaVu Serif and Sans Mono for vl-convert; the CI job renders one plot of each through the harness. The eval fixtures gain 120 cases and two smoke cases; the matrix harness warns when AGENT_LIBRARIES does not enable a selected case's library. Eligibility: plotnine 264 of 321, altair 269 of 325; matplotlib and seaborn unchanged against main. AGENT_LIBRARIES keeps its default until the owner's matrix run. Also folds the low findings of the step-1a review: the docs name every code artifact, the renderer's status row lists `languages`, a test pins the five copies of the artifact name table, and the limits prose says which rows run without RLIMIT_NPROC. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WbjVqBweiXjUZKp626ZFWm
… memory-floor test CodeQL flagged `_FENCE_TAG` on this PR (py/polynomial-redos, high): two blank runs around an optional slash are one ambiguous run, so `<` followed by a long run of blanks backtracked quadratically on user text. The blanks after the slash now sit inside the slash's own group; the matches are the same. The memory-floor executor test killed the run after three watchdog samples (60 ms), before a loaded machine's fake launcher had logged its `do`, and then compared the deletes with an empty launch log (CI on 63d98e0, and locally under -n 4). The floor is now crossed once the launch is in the log. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WbjVqBweiXjUZKp626ZFWm
Five tests reach the lines the suite covered only through a sandbox child: `make_backend`'s resolver hands the local backend the registry's runtime with the render timeout as its CPU limit, `opening.assess` keeps a library without a runtime as it is and blocks it as unsupported, `frame_for_readers` answers the caller's frame or None, `_renderer` attaches Agg's canvas to a bare figure, and `harness.main --exec` frames a child, maps a signal death and a missing command and refuses an empty command, with `set_limits` patched out of the test process. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WbjVqBweiXjUZKp626ZFWm
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Two commits, one per plan step; each reviews on its own (
agentic/specs/261024-agent-all-libraries.md). Say so if you want them as two PRs.agents/renderer/commands.py, keyed by the job's(language, library)and never taken from the request; the probe harness gains an exec mode (harness.py --exec -- <command>) that runs a non-Python command under the same limits, start and end lines, launcher retry and kill path;GET /statusreports the toolchains the image carries and a job in a language without one answers503 runtime_unavailable. On the agents side the registryagents/anyplot/render/runtimes/runtime_forhands the pipeline, the theme toggle, the local backend and the eval harness oneRuntimeAdapterper library, which also carries the language's code conventions, error grammar, export and the render time a repair reserves. The exported code's artifact name follows the language (plot.py,plot.R,plot.jl,plot.js,plot.tsx) through the wire, the BFF, the feedback case and the result card.regions.SAVE_RULES) replaces the savefig rule in the regions, the validator, the normaliser, the readiness scan and the edit applier (plotnine'ssaveandggsave, altair'ssavewith its secondplot-<theme>.htmland the PIL padding block). The validator gains both import profiles, the import surveys' bans and the routes their adversarial pass found (altair's image mark andfrom_dict, the narwhals frame behindlimit_rows,transformed_data, plotnine's.environment), parses plotnine's aesthetic and facet strings as code and takes them only as literals it can read, and refusesformula=andfrom_formula. The harness admits the HTML write, hands plotnine'sEnvironment.capturethe plot's frame, letsEnvironment.evalrun the walked strings, and attaches an Agg canvas so the probe measures plotnine's figure. The altair command row runs withoutRLIMIT_AS(vl-convert's V8 aborts under 64 GiB of address space). The renderer image installs both libraries and links DejaVu Serif and Sans Mono for vl-convert;ci-image.ymlrenders one plot of each through the harness. The eval fixtures gain 120 cases and two smoke cases.py/polynomial-redos(high) onpolicy._FENCE_TAG, whose two blank runs around an optional slash backtracked quadratically on user text; the blanks after the slash now sit inside the slash's own group, the matches are the same, and the CodeQL CLI (2.27.3, the workflow's suite andpaths-ignore) no longer reports it locally. And the memory-floor executor test, which killed the run 60 ms after the launch and then compared the deletes with a launch log the loaded runner's fake launcher had not written yet; the floor is now crossed once the launch is in the log.opening.assessfor a library without a runtime,frame_for_readers,_renderer, andharness.main --execwithset_limitspatched out (the exit codes, the start and end lines, the usage error). Patch coverage 96.4 %; what stays uncovered is the nine Protocol stubs incontract.pyand four harness lines only the sandbox child runs.Nothing changes for users until
AGENT_LIBRARIESnames plotnine or altair (the default staysmatplotlib,seaborn; the plan's 80 % matrix gate decides).Eligibility (
agents.evals.eligibility, 2026-10-25)The plan lists every blocked file's cause. A harness sweep with the test venv rendered 258 of the eligible plotnine files and 258 of the eligible altair files.
Plan
agentic/specs/261024-agent-all-libraries.mdTest plan
uv run ruff check agents api core tests,ruff format --check,uv run mypy api core agents,uv run python -m tools.changelog check --base origin/mainpassuv run pytest tests/unit/agents tests/unit/api tests/unit/core: 11,798 passed (thetests/unit/workflowsandagentic/regenfailures of the cloud container are environmental, as noted for 1a)cd app && yarn lint && yarn fm:check && yarn type-checkpass (1a); 1b changes no frontend file-n 4load after b182dd4AGENT_LIBRARIES=matplotlib,seaborn,plotnine,altair), then_LIBRARIESinagents/cloudbuild.yaml; the plan's open questions, or the defaults stand