Skip to content

feat(sandbox): infra images keep their build contexts, and a sandbox that can't run an image's tar.gz builds it from that - #169

Merged
earakely-scale merged 4 commits into
mainfrom
infra-build-contexts
Oct 12, 2026
Merged

earakely-scale merged 4 commits into
mainfrom
infra-build-contexts

Conversation

@earakely-scale

@earakely-scale earakely-scale commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

The infra images (the gateway, the service-db's three, the website browser) now keep the build context they're built from, and a sandbox that can't run an image's tar.gz builds it from that instead. So a bundle run from an arm64 Mac deploys its gateway on a remote amd64 VM or on Modal, which it couldn't before.

Why

The run builds the infra for this machine. From Apple silicon that's linux/arm64, and a remote VM runs linux/amd64, so the VM refused the image after it was created, and the deploy failed. Modal's container gateway refused the infra outright: it pulls images by name, and they're in this machine's registry. A run whose deploys were all remote didn't build the infra at all, and asked for a manual put.

What

  • A VM rebuilds an image whose tar.gz is for another platform. DockerImageArtifact.rebuilt_for(platform) is true for an image with both a tar.gz and a build context whose tar.gz the VM can't run. VmSandbox.load_docker_images builds those from the context, for the VM's own platform, and still loads every other tar.gz, so an image stays byte-identical wherever it can. The rule covers any image carrying both, which every put() with a build context writes.
  • Modal builds an image it can't run by name. Accepts.NAME_OR_CONTEXT takes any image with a build context, and ModalSandboxProvider.prepare_image builds one that is only a context, named for a registry Modal can't reach, or built for another platform than Modal's.
  • Modal's container gateway prepares its service-db images too. The stored service-db images it starts are checked and prepared like the gateway's and its servers', so EnvironmentGatewayProvider.accepts loses its prepared argument.
  • The infra puts keep their build contexts. One helper builds each infra image from its Dockerfile's folder and stores that folder as its build context. The gateway's context narrows to gateway/ (its COPY paths adjusted). build_inputs_digest now hashes those build contexts rather than a hand-kept file list, so what a run records is what a sandbox rebuilds from. The website browser's Dockerfile gets ARG PLAYWRIGHT_MCP_VERSION=0.0.41, because a context carries no build args, and is now the only place the version lives: the PLAYWRIGHT_MCP_VERSION Python constant, which nothing read any more, goes. db-mcp drops its RUN --mount cache, which needs BuildKit (modal_vm's Docker has none) and does nothing in a fresh build, pins its git clone to 15c8e333 (upstream HEAD), and fails its build if its sed patch stops applying.
  • A run builds the infra for remote deploys too, in local stores. Preflight adds a gateway deploy's infra kinds when any of its providers takes it and the stores are local, refuses a missing infra env only for stores that aren't local, and refuses infra it would build with no Docker here.

The Dockerfile edits, and hashing the build contexts, change build_inputs_digest, so a run rebuilds any stock-built infra in local stores once, now with its build context. Infra in other stores is unchanged until someone puts it again.

Tests

Slow integration tier (public, no secrets): the local sandbox, given this machine's platform as a remote VM records its own, builds an image that is only a build context, and rebuilds one whose tar.gz records another platform from the context beside it (that tar.gz points nowhere, so loading it would fail the deploy); each serves its card through the gateway. No integration test covered building from a build context before. A unit test checks every shipped infra Dockerfile builds from its context alone on Docker's legacy builder (no BuildKit-only syntax, no build arg without a default), which would have caught db-mcp's RUN --mount.

Unit tier: 7,457 passed. New: a VM building from the context beside a tar.gz of another platform and loading one of its own; rebuilt_for and load_problem for each case; Modal building a tar.gz-plus-context image named for this machine's registry or another platform, and pulling one it can; Accepts for that shape; the container gateway preparing its service-db images; each infra put keeping its context, the one its build inputs cover; the build inputs covering every file in a build context but Python caches; preflight building infra for a remote deploy in local stores, refusing it elsewhere, and refusing it with no Docker; stock infra built from other inputs rebuilt for Modal rather than refused as it stands; a pulled image dropping one Modal prepared earlier under its name.

Live checks

From an arm64 Mac with fresh local stores, a bundle whose MCP server deploys through the gateway with local Postgres state, checked by a verifier calling it through the gateway. Every run passed:

  • local: passed in 45 s. The run built the gateway and service-db here (linux/arm64, now with their build contexts), and the local deploy loaded them as before.
  • modal_vm, after that: passed in 139 s. The VM rebuilt each arm64 infra image from its build context, db-mcp included, with Docker's legacy builder. Before this change the VM refused them.
  • Modal, after that: passed in 50 s. Modal built the gateway and service-db images from their build contexts. Before this change it refused them, since they're in this machine's registry.
  • modal_vm, Modal and Sail, each from fresh stores: passed in 146 s, 22 s and 165 s. The run built the infra here for deploys that were all remote, which it used to refuse, and each sandbox rebuilt it.
  • After the simplification commit, from fresh stores: local twice, then Modal, passed in 49 s, 30 s and 11 s. The second local run built no infra, so what a put records matches what the next run computes.
  • Chaos, from this arm64 Mac and from an amd64 EC2 host: fresh stores on modal, modal_vm, sail_vm and local; no Docker (refused before any write); a corrupt, missing or failing gateway build context (fails naming the image; on amd64 modal_vm loads the tar.gz and never reads it); a failing env build; a Sail outage falling back to modal_vm and Modal; four concurrent runs (infra built once, the others wait); Ctrl-C and SIGTERM mid-deploy; infra built by 0.9.1315 rebuilt once with its contexts; a 6-minute soak. Every run that should pass passed.
  • Nothing was left running on Modal or Sail, except two Modal containers from one run cancelled 6 s in, which I terminated: a cancel during the container gateway's parallel creates loses the containers already made. That code predates this PR.

Cold builds in a modal_vm VM (2 vCPU, 8 GB): gateway 20 s, service-db 4 s, db-web 2 s, website browser 57 s. db-mcp failed there before this change, since RUN --mount needs BuildKit.

🤖 Generated with Claude Code

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge, with both earlier findings fixed and no new actionable issues.

Summary

Infra images now keep their build contexts. VMs and Modal can rebuild images they cannot run as stored. Local stores also build missing infra for remote deployments.

  • VMs rebuild images from another platform for their own platform.
  • Modal builds images its containers can’t pull or run.
  • Infra images travel with the folders that can rebuild them.
  • Remote deployments can build missing infra from local stores.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Stored infra image] --> B{Sandbox can run stored image?}
  B -->|Yes| C[Load tarball or pull image]
  B -->|No, build context available| D[Build from saved context]
  D --> E[Run on sandbox platform]
  C --> E
Loading

Reviews (4) · Last reviewed commit: "test: a VM builds images from their buil..." · Reviewed by Greptile

…that can't run an image's tar.gz builds it from that

The run builds the gateway and service-db for this machine, linux/arm64 on Apple silicon, so a remote amd64 VM
refused them after it was created, Modal's container gateway refused them as images in this machine's registry, and a
run whose deploys were all remote didn't build them at all.

- DockerImageArtifact.rebuilt_for(platform): an image with a tar.gz and a build context whose tar.gz a VM of that
  platform can't run. VmSandbox.load_docker_images builds those from the context for the VM's own platform and loads
  every other tar.gz, so an image stays byte-identical wherever it can.
- Accepts.NAME_OR_CONTEXT takes any image with a build context, and Modal's prepare_image builds one that is only a
  context, named for a registry Modal can't reach, or built for another platform than Modal's.
- Modal's container gateway checks and prepares its stored service-db images like its other images, so
  EnvironmentGatewayProvider.accepts loses its prepared argument.
- The infra puts keep their build contexts. The gateway's narrows to gateway/, the website browser's Dockerfile
  carries PLAYWRIGHT_MCP_VERSION's default (a context carries no build args), and db-mcp drops its BuildKit-only cache
  mount, pins its clone, and fails if its sed patch stops applying.
- With local stores, a run builds the infra for remote deploys too, and refuses it up front with no Docker here.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Comment thread src/agent_env/bundle/preflight.py
Comment thread src/agent_env/providers/sandbox_providers/modal_sandbox.py Outdated
earakely-scale and others added 3 commits October 11, 2026 21:47
…dal drops a prepared image it then pulls

Stock infra in local stores built before its images kept their build contexts is in this machine's registry, so on
Modal preflight refused it and never scheduled the rebuild that would fix it; infra the run builds anew (missing, or
built from other inputs) is no longer checked. And a Modal provider that prepared an image under a name, then a
pullable one under the same name, built the first: prepare_image now drops the earlier entry when it pulls.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…uild inputs are its build contexts

- `_build_and_put` builds an infra image from its Dockerfile's folder and stores that folder as its build context,
  so the context a sandbox rebuilds from can't drift from what was built here.
- `build_inputs_digest` hashes those build contexts (`BuildContext.source_digest`) instead of a hand-kept file list,
  so a Dockerfile that copies a new file is rebuilt without anyone updating the list.
- The website browser's Dockerfile `ARG` is its one Playwright MCP version; the unread Python constant goes.
- Preflight and `ensure_default_envs` share one message for infra that needs docker.
- Modal's `prepare_image` drops a check `by_name_problem` already makes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…fra Dockerfiles build on Docker's legacy builder

- Slow integration tier, local backend: the local sandbox, given this machine's platform as a remote VM records its
  own, builds an image that is only a build context, and rebuilds one whose tar.gz records another platform from the
  context beside it (its tar.gz points nowhere, so loading it would fail the deploy). Each serves its card through the
  gateway. The first integration coverage of a sandbox building from a build context.
- Unit: every shipped infra Dockerfile builds from its context alone on Docker's legacy builder, which a VM with no
  buildx uses: no BuildKit-only syntax, no build arg without a default.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@earakely-scale
earakely-scale merged commit 68fad07 into main Oct 12, 2026
14 checks passed
@earakely-scale
earakely-scale deleted the infra-build-contexts branch October 12, 2026 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant