ci: give task lint a 10-minute timeout in the tend sandbox

[prql-bot]

A tend session that runs task lint today sees it killed partway through, and can't tell that from the gate being broken.

pre-commit builds its hook environments into ~/.cache/pre-commit, which lives in the sandbox home and is discarded with the session — so the cold build is paid by every session that lints, not just the first. It measured 233.7 s in run 35321455559, against the agent's 120 s default command timeout; the cargo fmt hook on its own ran past 300 s there on a cold rustup.

600 s is the agent's maximum for a single command, not a floor to raise — and task lint is pre-commit run --all-files followed by a whole-workspace --all-targets --all-features clippy, which on a cold cargo cache is the larger half. So the note names 600 s as a ceiling and gives the fallback of running the two commands separately, each with its own 600 s, rather than leaving a session to read the kill as a broken gate.

That matters because #6334 is restoring task lint to the sandbox by installing pre-commit and the pinned toolchain. Without a note telling a session to extend the timeout, the first thing it does with the restored gate is watch it get killed — which lands back at the nightly's "task lint couldn't run", from a different cause. This adds the note so the fix in #6334 is usable; it's inert until that lands.

running-tend rather than the bundled skill: the numbers and the cause are this repo's — its hook set, its sandbox — and .config/tend.yaml, which is where the cost is currently recorded, isn't read at session start.

[prql-bot]

The 10-minute figure is derived from the pre-commit hook build alone, but task lint is pre-commit run --all-files then cargo clippy --all-targets --all-features. Two consequences:

• 600 s is the harness's maximum for a single command, so "at least 10 minutes" asks for headroom that doesn't exist — a session reads it as a floor and there's nothing above it.
• The clippy half is a whole-workspace --all-targets build. The Verifying a rust-toolchain.toml bump section directly below records that shape as exceeding the session budget on a cold cargo cache, with a single scoped compilation unit already at ~9 minutes. A session that sets 600 s and still gets killed — in clippy this time, with the hook build's ~4 minutes already spent — lands back at "task lint couldn't run", which is the outcome this section exists to prevent.

Suggestions inline: name 600 s as the ceiling rather than a floor, and give the fallback of running the two commands separately, each with its own 600 s.

[max-sixty]

so there is no caching available in the tend sandbox? that's a shame if so; makes runs much longer

is there a way around this or no?

[prql-bot]

Confirmed — and it's structural rather than a missing knob. The agent no longer works in the runner's checkout; it gets a disposable clone under /tmp, with denyRead on both the runner's home and the runner's checkout. So an Actions cache restore in setup: lands where the agent can't read it, and nothing inside the sandbox can call the cache API itself — only GITHUB_* names cross into the sandbox environment, and per the upstream analysis the cache service's credentials aren't GITHUB_-prefixed. It's open upstream as max-sixty/tend#1198, filed on exactly this symptom from another repo; the candidate fixes are all "tend relocates runner-prepared paths into the agent's tree", and none is built yet.

Measured from inside this session: ~/.cargo has no registry/, the clone has no target/, and /home/runner is an empty directory. So every cargo invocation in a tend session compiles cold — that's the larger half of the cost, and pre-commit the smaller one.

Two things fall out of that, both worth fixing here rather than waiting on upstream:

• Our tend-setup cache step has been restoring nothing at all, independent of the boundary: its prefix-key omits the ${{ env.version }} component that the saving jobs include, so the key can't match one that was saved, and all three tend runs I checked log No cache found.. Since a hit would be unreadable anyway, ci: remove tend-setup steps that don't reach the tend sandbox #6336 drops it along with the set_version.sh step that exists only to feed its key — ~8.6 s per job across all eight tend-* workflows.
• The RUSTFLAGS/RUSTDOCFLAGS writes in that same action don't reach the agent either — $GITHUB_ENV doesn't cross the boundary, and both read as unset in-session. sandbox_env: in .config/tend.yaml sets them for real, and -C debuginfo=0 on permanently-cold builds is the one free speedup available today. I left it out of ci: remove tend-setup steps that don't reach the tend sandbox #6336 because it trades away line numbers in backtraces; happy to add it if you want it.

What is available for warming is sandbox_setup: — it runs as the sandbox user before the agent starts, isn't bound by the agent's 120 s command timeout, and what it writes to the sandbox home lasts the session. pre-commit install-hooks there would move the hook build (3m40s, measured in this session) out of the agent's budget entirely, at the price of every session paying it including the many that never lint — which is why #6334 left it to first use. cargo fetch is the same trade for the registry. Neither touches target/, which is the expensive part and the part only #1198 can fix.

This PR is still worth having either way: as long as the hook build happens inside the session, a session needs to know a multi-minute task lint isn't a broken gate.

[max-sixty]

would there be a way to run in our normal path, with access to caches etc; without giving up access to standard GHA caching?

[prql-bot]

Not by configuration, no — but the two halves of the question come apart, and only the first one is actually blocked.

Standard GHA caching isn't at stake. tests.yaml, test-rust.yaml and the rest keep their Swatinem/rust-cache entries regardless; the shared rust-x86_64-unknown-linux-gnu cache on main is 1,951 MB. The only broken consumer was tend-setup's own restore, unmatchable and unreadable, and #6336 has since removed it. So there's no trade here — nothing gets given up either way.

Running in the runner's checkout isn't a knob. On 0.2.10 (released yesterday, current) prepare_agent_workspace.py unconditionally clones to /tmp, and sandbox_runtime.mjs hard-codes denyRead: [runnerHome, runnerWorkspace]. The whole adopter surface is setup:, sandbox_path:, sandbox_env:, sandbox_setup: and JSON-merge-patch overrides on the generated YAML — and step-level overrides are explicitly unsupported, so there's no post-step to save a cache from either. Getting back to the runner checkout means pinning back to the 0.2.0-era action (the disposable workspace landed between 0.2.0 and 0.2.5) or forking, which hands the agent the tree holding the runner's orchestration state. That's the thing the boundary exists for; I wouldn't.

What is reachable inside the boundary is artifacts, not the cache API. I measured this in this session rather than reasoning about it:

• gh run download works in-sandbox — the proxy injects the bot credential for api.github.com and the blob fetch is presigned. A 24 MB artifact took 2.0 s (~12 MB/s), so a rust-cache-sized payload is ~3 minutes, and sandbox_setup: runs outside the agent's 120 s command timeout.
• Cargo tolerates the relocation. Built a scratch crate, tarred the tree (including target/) to a different absolute path, rebuilt: Finished in 0.00s, nothing recompiled. So the random /tmp/tend-agent-workspace-*/checkout path is not the problem I expected it to be.
• Cargo does not tolerate a relocated CARGO_HOME. Same tree, registry copied elsewhere, CARGO_HOME pointed at the copy: Compiling itoa, Compiling a — registry units rebuild, and everything above them with it.

That last one is the real constraint: a warming job has to produce the tree with CARGO_HOME=/home/tend-sandbox/.cargo and matching RUSTFLAGS, because the sandbox home is the only path that's both writable and stable across sessions. And setup: can't stage into it — the sandbox user is created by the action at ensure_sandbox_user(), after setup: steps have run.

So the warm-target/ path is a thing to build, not a setting to flip: a scheduled job that builds under sandbox-user paths, tars target/ plus ~/.cargo, uploads it, and one sandbox_setup: line that unpacks the latest. ~2 GB each way per day, paid by every session including the many that never compile. Say the word and I'll build it; I don't think it's obviously worth it.

Two things that are cheap and available now, independent of that:

• cargo fetch in sandbox_setup: — registry only, no target/, download cost each session.
• sandbox_env: {RUSTFLAGS: "-C debuginfo=0"} — RUSTFLAGS reads unset in-session (the runner-side $GITHUB_ENV write never crossed, and ci: remove tend-setup steps that don't reach the tend sandbox #6336 removed it), so every permanently-cold build in a tend session carries full debuginfo it never uses. sandbox_env: is the only place that actually sets it.

Upstream, max-sixty/tend#1198 option 1 is still the clean fix, and the CARGO_HOME result is a constraint on its shape — the hand-off has to land ~/.cargo in the sandbox home, not just the workspace. I can add that measurement to the issue if useful.

Commands behind the measurements

# artifact throughput, from inside the session
gh run download 35333728039 -n prqlc_c-aarch64-apple-darwin-dev -D "$TMPDIR/arttest2"
# real 0m1.980s, 24M unpacked

# relocation is fine
cargo new a && cd a && echo 'itoa = "1"' >> Cargo.toml && cargo build
mkdir ../b && (cd ../a && tar cf - .) | (cd ../b && tar xf -)
cd ../b && cargo build   # Finished in 0.00s

# relocated CARGO_HOME is not
(cd ~/.cargo && tar cf - registry) | (cd "$TMPDIR/cargohome2" && tar xf -)
CARGO_HOME="$TMPDIR/cargohome2" cargo build   # Compiling itoa, Compiling a

Cache size: gh api "repos/PRQL/prql/actions/caches" → 1,951 MB for <version>-<Cargo.lock hash>-rust-x86_64-unknown-linux-gnu on refs/heads/main.