test(runtime): verify remote MicroSandbox agent binary delivery

[skevetter]

Remote providers need evidence that runtime preparation happens inside the remote agent and that a verified workspace binary remains usable when its download source is unavailable. This adds one Linux/amd64 KVM scenario through ordinary up with the published MicroSandbox 0.1.8 provider and agent.local: false.

The owned container installs the system CA bundle and checks it during preflight so ordinary HTTPS image metadata resolution has its normal trust roots. Fixture listener ports are normalized for IPv4 and IPv6 wildcard addresses before building Docker bridge URLs. A fresh owned container uses separate fixture endpoints for the Devsy agent and Agent.Binaries runtime. Independent counters require an agent executable transfer and exactly one checksum-verified runtime download. After successful guest access, the test removes only the agent's private provider-download cache, makes the runtime endpoint return 404, and stops/restarts the workspace. The runtime request count must stay unchanged, the workspace binary must retain its checksum, executable mode and inode, and a guest command must still succeed. The agent executable endpoint stays available; this does not claim offline reuse of that separate binary. Workspace, container and server cleanup have independent bounded attempts.

The branch integrates the actual merged consumer/Go CI foundation. One external-only catalogue row brings the total to 15 suites/78 cases (37 built-in, 41 external). Existing runner/report validation, Windows dynamic coverage, caching and independent deadlines remain unchanged. No production behavior or default provider changes.

Local validation: complete workspace/helper race tests, Linux E2E/helper vet, Linux amd64/arm64 E2E compilation, isolated Linux endpoint unit execution (including IPv4/IPv6 listener-address controls) and module verification passed. The one-case Ginkgo selection was a dry run only. Strict Linux lint passed with zero issues, and all applicable scoped hooks completed; all 13 affected-file hooks passed or had expected no-file skips. An isolated Ubuntu-container Go HTTPS check reproduced missing trust roots and succeeded after installing the system CA package; it does not prove native VM behavior. Local CodeRabbit remains temporarily waived by the user while its service cannot start; no local review coverage is claimed.

Exact-head implementation run 38612845492 passed all 87 jobs. All 15 native reports passed once: 78 selected cases (37 built-in, 41 external), including actual remote-agent delivery and workspace-copy reuse after the runtime source returns 404, with no selected skips, dry run or focus. Windows passed all 42 dynamically discovered roots and 84 test/subtest IDs plus both Go helper packages. The current real-provider performance artifact records successful fresh/reused up and 808 reaped warm probes; it makes no cold-cache or speedup claim. Netlify canceled unchanged-content builds with successful GitHub statuses; no build failure was waived. Fresh Greptile completed for this exact head with confidence 5/5 and no actionable findings. The explicitly requested full CodeRabbit review completed while draft for all five files with no actionable findings, Minimal merge risk and Low security architecture risk; retained concerns and all review endpoints were inspected. Final current-head checks, threads and all twelve GitHub signatures were refreshed before readiness. External-provider transition remains gated on migration and applicability work.

[netlify]

✅ Deploy Preview for devsydev canceled.

Name	Link
🔨 Latest commit	35c518f
🔍 Latest deploy log	https://app.netlify.com/projects/devsydev/deploys/6acfc57d05fc85884076e6ca

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 64f1de86-1103-486f-b37c-0beab4542f97

📥 Commits

Reviewing files that changed from the base of the PR and between 904c504 and 35c518f.

📒 Files selected for processing: 5

• .github/workflows/pr-ci.yml
• e2e/tests/up/MICROSANDBOX.md
• e2e/tests/up/provider_microsandbox_remote_agent_linux.go
• e2e/tests/up/provider_microsandbox_remote_agent_linux_test.go
• sites/docs-devsy-sh/content/docs/developing-providers/runtime-protocol.mdx

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 1 remain after this review.

📝 Walkthrough

Adds a Linux-only MicroSandbox external-provider end-to-end test for remote agent binaries. The test checks artifact downloads, runtime binary integrity, and workspace runtime reuse after the provider-download cache is cleared and the runtime endpoint returns 404.

Changes: Remote agent binary coverage

Layer / File(s)	Summary
Linux and Docker test prerequisites e2e/tests/up/provider_microsandbox_remote_agent_linux.go	Adds host and container prerequisite checks, container setup, and bounded preflight and cleanup helpers.
Artifact fixture and endpoint tests e2e/tests/up/provider_microsandbox_remote_agent_linux.go, e2e/tests/up/provider_microsandbox_remote_agent_linux_test.go	Adds HTTP fixture endpoints for agent and runtime artifacts, tracks GET requests, supports offline runtime responses, and tests URL and endpoint behavior.
Remote binary download and reuse verification e2e/tests/up/provider_microsandbox_remote_agent_linux.go, .github/workflows/pr-ci.yml, e2e/tests/up/MICROSANDBOX.md, sites/docs-devsy-sh/content/docs/developing-providers/runtime-protocol.mdx	Adds the end-to-end checks for artifact downloads and workspace runtime reuse. Registers the case in CI and documents its checks and catalogue count.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~30 minutes

Change: Other

Sequence Diagram(s)

sequenceDiagram
  participant Test as Remote agent binary test
  participant Provider as Provider container
  participant Fixture as Artifact fixture
  participant Workspace as Workspace runtime
  Provider->>Fixture: GET agent and runtime artifacts
  Test->>Provider: Clear provider-download cache and stop workspace
  Test->>Fixture: Set runtime endpoint offline
  Test->>Provider: Start workspace
  Provider->>Workspace: Reuse retained runtime binary
  Test->>Workspace: Run guest SSH check

Loading

Merge Risk: ⚪ Minimal · up to 35c51

This change adds test coverage and documentation only, with no production behavior change. No concrete merge-blocking risk was identified in the reviewed files.

Security architecture risk: 🔵 Low · up to 35c51

The change is confined to a temporary test environment. Fixed artifact responses, runtime checksum verification, and independently bounded cleanup limit exposure. Runner network isolation and cleanup after abrupt termination are not established; no introduced security vulnerability was verified.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

• inferred — Compromise of executable delivery into the fixture container could reach the Docker host because its socket is mounted. Merely reaching the artifact HTTP handlers does not confer that authority: requests cannot replace the fixed payloads or invoke container commands. Reachability beyond the runner is not established.

Trust Boundaries and Controls

• observed — The unauthenticated HTTP fixture exposes two fixed GET routes, rejects other methods, and uses HTTP timeouts. Runtime-offline state changes only the runtime response; the agent endpoint stays available. Handler tests check method rejection and independent request counters.

Resilience and Maintainability Implications

• observed — Resources use generated container and private-home names. Workspace, container and server cleanup have independent bounded attempts that do not inherit an expired spec context. Container removal tolerates an already-absent container, and cache deletion is shell-quoted and restricted to the private provider cache. These controls do not guarantee cleanup after process kill or runner termination.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 20 functions across 2 files. (3 skipped: 3…	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly identifies the test change and matches the main focus on remote MicroSandbox agent binary delivery.

Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 20 functions across 2 files. (3 skipped: 3 unsupported.)

• Fix all pre-merge checks with AI

✨ Finishing Touches

✨ Simplify code

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[netlify]

✅ Deploy Preview for images-devsy-sh canceled.

Name	Link
🔨 Latest commit	35c518f
🔍 Latest deploy log	https://app.netlify.com/projects/images-devsy-sh/deploys/6acfc57d18c956fc66d5dc52

[skevetter]

@greptileai review

[greptile-apps]

Confidence Score: 5/5

[Medium impact] No actionable code defect was found; the changes appear safe to merge once the required native CI checks pass.

Summary

Adds a Linux/KVM test for remote MicroSandbox runtime delivery and workspace-binary reuse.

• A Linux test checks remote runtime delivery and workspace reuse.

Diagram

sequenceDiagram
  participant Test
  participant Fixture
  participant Agent as Remote agent container
  participant Guest
  Test->>Agent: Ordinary workspace up
  Agent->>Fixture: Download Devsy agent
  Agent->>Fixture: Download provider runtime
  Agent->>Guest: Start workspace
  Test->>Guest: Check command succeeds
  Test->>Agent: Check binary and clear provider cache
  Test->>Agent: Stop workspace
  Test->>Fixture: Make runtime endpoint return 404
  Test->>Agent: Start workspace again
  Note over Agent: Retain workspace runtime binary
  Test->>Agent: Check checksum, mode, and inode
  Test->>Guest: Check command succeeds

Loading

_{Reviews (1) · Last reviewed commit: "fix(e2e): prepare system trust for remot..." · Reviewed by Greptile}

[skevetter]

@coderabbitai full review

Please review the complete current draft PR at 35c518f, including all five changed files and retained security, architecture, cleanup/recovery and merge-risk concerns. Exact-head implementation CI and all 78 native scenarios passed; Greptile completed 5/5. Keep this PR draft while the full review completes.

[coderabbitai]

✅ Action performed

Full review finished.

[skevetter]

Final review disposition for 35c518fa3094fffb320f0cc4c07715aae244fccc:

• Greptile completed for this head with confidence 5/5 and no actionable findings. CodeRabbit full review 64f1de86-1103-486f-b37c-0beab4542f97 completed for all five changed files; its entire retained security/architecture and merge-risk assessment, summary and empty inline/thread endpoints were inspected. It reports Minimal merge risk, Low security architecture risk and no architecture-level concerns.
• The retained Docker-socket blast-radius and abrupt-termination cleanup limits are accurate fixture boundaries. The container needs the existing Docker/KVM path; artifact handlers return fixed tested-checkout agent bytes and checksum-verified published runtime bytes, accept no mutation/upload/command requests, and the fixture owns generated resources with independent bounded cleanup. This does not claim network isolation or cleanup after runner termination. No new production permission or rollback guarantee is introduced.
• The docstring-percentage warning is not a missing public API contract: these are private helpers and conventional Go test functions in the E2E test package. The guide and protocol page describe prerequisites, exact download/reuse assertions, cleanup ownership and evidence limits. Adding comments that repeat each helper name would not improve that contract; no boilerplate is added solely to satisfy this heuristic.
• Exact-head implementation run 38612845492 passed all 87 jobs. All 78 selected native cases passed once, including remote-agent download and retained workspace-runtime reuse. Windows passed all 42 discovered roots/84 test IDs and both helper packages. Performance records warm observations only. Both Netlify deployments explicitly canceled because their base-directory content was unchanged, with successful status contexts; no failing external check was waived. All twelve commits have valid GitHub signatures. Local CodeRabbit service remains temporarily waived by the user; no local review coverage is claimed.