Skip to content

CI: hermetic spawns in NuGet/PyPI suites (#823) - #1498

Merged
Mikola Lysenko (mikolalysenko) merged 3 commits into
mainfrom
agent/fix-ci-hermetic-nuget-pypi
Oct 13, 2026
Merged

Mikola Lysenko (mikolalysenko) merged 3 commits into
mainfrom
agent/fix-ci-hermetic-nuget-pypi

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Oct 13, 2026 •

Copy link
Copy Markdown
Collaborator

LLM Description written by Claude Code:claude-opus-5-5

Refs #823 (slice 12; the issue stays open for the remaining PENDING_RAW_SPAWNS files)

What

The four NuGet / PyPI suites still on PENDING_RAW_SPAWNS started socket-patch with a bare Command::new and their own partial SOCKET_* handling, which had drifted from tests/common/hermetic.rs:

  • e2e_nuget.rs: removed six named vars. Any other ambient SOCKET_* reached the child. With SOCKET_ECOSYSTEMS=npm in the shell, scan_discovers_legacy_packages and scan_discovers_global_cache_packages fail on main (No npm packages found.). Nothing forced the config layer or update notifier off either. Its scans are unscoped, so an activated venv or the developer's HOME (~/.m2, user site) could also change the counts these tests assert.
  • e2e_nuget_dotnet_build.rs: had a private sweep plus hand-set opt-outs, with no update-check opt-out.
  • e2e_pypi.rs: removed the flag-bound vars by name, but every other SOCKET_*, the opt-outs and VIRTUAL_ENV were unhandled.
  • e2e_pypi_multi_copy.rs: had a private sweep plus hand-set opt-outs, with no telemetry opt-out. It keeps its own Python DISCOVERY_VARS scrub.

All four now spawn through hermetic::command, and the three that run unscoped scans add Extra::Venv. e2e_nuget and e2e_pypi also call jvm_env::isolate_cli, the stand-in HOME that common::run_bin_with_env already uses. The files leave PENDING_RAW_SPAWNS (26 -> 22), so the spawn_env_hygiene ratchet keeps them hermetic.

SOCKET_SAVE_ONLY joins HOSTILE_SEEDS, carrying over the seed e2e_pypi used to set itself. command_never_leaks_its_hostile_seeds now iterates HOSTILE_SEEDS instead of a hand copy, so a new seed can't go unchecked (found by /code-review high).

Test-only change; no product code touched. No file overlap with #1495, which removes other lines from the same PENDING_RAW_SPAWNS list. Whichever PR lands second needs a trivial rebase of that list.

CI cost

No workflow change. A lean PR run keeps the same job count and job-minutes before and after, because these suites already run in the same jobs.

Test evidence

  • cargo fmt --all -- --check: clean.
  • cargo clippy -p socket-patch-cli --all-features --tests: identical warning set to origin/main (pre-existing duplicate_mod / needless_borrow / type_complexity in test code).
  • Red on main: with SOCKET_ECOSYSTEMS=npm SOCKET_DRY_RUN=true SOCKET_CLI_API_TOKEN=x SOCKET_NO_CONFIG=0 VIRTUAL_ENV=<venv with six> exported, cargo test -p socket-patch-cli --all-features --test e2e_nuget -- --include-ignored gives 20 passed, 2 failed (scan did not print a Found N packages line; output was: No npm packages found.).
  • Green on this branch, same hostile env plus SOCKET_SAVE_ONLY=true, -- --include-ignored: e2e_nuget 22/22, e2e_pypi_multi_copy 3/3, e2e_nuget_dotnet_build 22/22 (no local .NET SDK, so the SDK legs skip; lean CI's .NET 10 leg runs them), spawn_env_hygiene 15/15, e2e_pypi 20 passed. The 5 live-API e2e_pypi tests fail here only because the sandbox proxy blocks patches-api.socket.dev. The error shows the child correctly chose the public proxy despite the ambient SOCKET_CLI_API_TOKEN. CI doesn't run those tests either (ci.yml: live-API smoke suites are on demand only).

/code-review high

Fixed: the seed-leak test not checking the new seed, e2e_pypi not pinning HOME, and the mixed common::hermetic_command / hermetic:: naming.
Not fixed:

  • Folding e2e_pypi_multi_copy's DISCOVERY_VARS (CONDA_PREFIX, WORKON_HOME, PIPENV_*, PYTHON*) into Extra::Venv. That widens a scrub used by about 20 other suites, some of which set those vars themselves, so it belongs in its own change.
  • Routing the two small per-file run helpers through common::run_with_env. The shared runner takes no Extra scrubs, and adding a parameter touches every caller.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DhzwitYXkXnY84GVNR1PgP


Generated by Claude Code

Assisted-by: Claude Code:claude-opus-5-5
The NuGet and PyPI e2e suites started socket-patch with a bare
Command::new and their own partial SOCKET_* scrubs, so a developer's
ambient SOCKET_* settings, real `socket login` config or activated
venv could change what the tests exercised. They now spawn through
the shared hermetic builder, scrub VIRTUAL_ENV, and leave the
spawn-hygiene ratchet's pending list (26 -> 22).

Refs #823

Assisted-by: Claude Code:claude-opus-5-5
The seed-leak test kept its own copy of the hostile seed list, so the
new SOCKET_SAVE_ONLY seed went unchecked; it now iterates the shared
list. The PyPI lifecycle runner also points HOME at the stand-in like
the shared runner, so the developer's user site stays out of scans.

Refs #823

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko
Mikola Lysenko (mikolalysenko) marked this pull request as ready for review October 13, 2026 19:05
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit a62c897. Configure here.

@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Oct 13, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Ready for review (burn-down agent)

  • Head: a62c8971167f6490e58c1771a48bdb4b49f6b218
  • CI: 86/86 settled green on this head (35 success, 50 path-skipped, 1 neutral); no main-wide failures
  • Bugbot: reviewed a62c897 and found no issues; no open review threads
  • Mergeable: clean, and it already has one human approval on this head
  • What to look at: the test-only change moves 4 NuGet/PyPI e2e suites onto hermetic::command (PENDING_RAW_SPAWNS goes from 26 to 22), and SOCKET_SAVE_ONLY is added to HOSTILE_SEEDS

Generated by Claude Code

Merged via the queue into main with commit 422e8c6 Oct 13, 2026
86 checks passed
@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the agent/fix-ci-hermetic-nuget-pypi branch October 13, 2026 22:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants