Conversation
The SDK's first-run Mutex needs a writable /tmp, which the sandbox doesn't have, and MSBuild's worker nodes fail silently because AF_UNIX binds are refused. Add a wrapper that runs a command with a private tmpfs at /tmp, and record both constraints plus the full test-dotnet recipe.
prql-bot
left a comment
There was a problem hiding this comment.
The /tmp premise doesn't reproduce. From this session, with /tmp still read-only, the whole test-dotnet job runs green without the wrapper — -m:1 is the only thing it needs.
Before concluding that, I cleared everything that could have been masking a first-run failure: ~/.dotnet, ~/.nuget, and both projects' bin/ and obj/. The configurer genuinely re-ran on the next command — it reprinted the Welcome to .NET 10.0! banner and recreated 10.0.400.dotnetFirstUseSentinel — so this is the cold path the new section describes, not a warm one:
$ dotnet build prqlc/bindings/dotnet -m:1 # no wrapper
Build succeeded.
$ cp target/debug/libprqlc_c.* prqlc/bindings/dotnet/PrqlCompiler{,.Tests}/bin/Debug/net10.0/
$ dotnet test prqlc/bindings/dotnet -m:1 # no wrapper
Passed! - Failed: 0, Passed: 10, Skipped: 0, Total: 10dotnet new console succeeds outside the wrapper too. None of the four commands the section says abort produced mkdtemp("/tmp/.dotnet.XXXXXX") == nullptr; errno == EROFS, and touch /tmp/... still fails with Read-only file system, so this isn't a sandbox that quietly gained a writable /tmp.
The -m:1 half reproduces exactly as described, and deterministically:
$ dotnet build prqlc/bindings/dotnet # no -m:1, no wrapper
Build FAILED.
0 Warning(s)
0 Error(s) # exit 1I can't reconcile the EROFS this description reports against a sandbox that was the same one ten minutes earlier, so I'm not claiming the authoring session misread it. What I can say is that nothing in this job needs a writable /tmp now, and that the silent node failure alone accounts for every symptom I can still reproduce.
That matters because the wrapper isn't free. Merging it tells every future .NET session to run builds under unshare --map-root-user --mount, which changes the build's effective uid to 0 and relocates the checkout to a bind path — MSBuildStartupDirectory and every absolute path in the build output come out as $TMPDIR/with-writable-tmp-checkout/... rather than the real checkout. That's a real behavior change to carry for a constraint I can't observe.
So I'm dropping with-writable-tmp.sh and the /tmp section, and keeping the -m:1 guidance and the reproduction recipe — the parts that hold up under a clean re-run. The script stays reachable in this PR's history if the EROFS ever reappears.
Full verification sequence
Each step ran from the repo root in this session, with /tmp read-only throughout:
touch /tmp/probe→Read-only file system. Confirms the sandbox constraint is still in place.- Wrapper path, as the PR documents it:
dotnet build prqlc/bindings/dotnet -m:1throughwith-writable-tmp.sh→Build succeeded. The wrapper itself works as described, and its namespace grants nothing new (mount -o remount,rwrefused; writing outside the checkout stillEROFS). rm -rf ~/.dotnet ~/.nuget prqlc/bindings/dotnet/*/{bin,obj}, thendotnet build prqlc/bindings/dotnet -m:1with no wrapper → first-run banner printed,Restoredboth projects,Build succeeded.cargo build -p prqlc-c, copylibprqlc_c.*into bothbin/Debug/net10.0/, thendotnet test prqlc/bindings/dotnet -m:1with no wrapper →Passed! - Failed: 0, Passed: 10.- Control, no wrapper and no
-m:1, twice from a cleanedbin//obj/→Build FAILED./0 Error(s)/ exit 1 both times; adding-m:1back →Build succeeded. dotnet new consolein a scratch directory, no wrapper →Restore succeeded.
The Newtonsoft.Json 9.0.1 NU1903 advisory the description mentions shows up on every one of these restores, so it's independent of any of this.
The read-only /tmp constraint the wrapper worked around does not reproduce: from a cleared ~/.dotnet, ~/.nuget and bin/obj, the whole test-dotnet job runs green without it (10 passed). MSBuild's silent node failure is the only real blocker, and -m:1 alone clears it.
358ca86 dropped with-writable-tmp.sh on the premise that a read-only /tmp no longer blocks the SDK. It does. From a fresh sandbox with no dotnetFirstUseSentinel: dotnet new console -> mkdtemp("/tmp/.dotnet.lDmpY9") == nullptr; EROFS dotnet build … -m:1 -> same failure, in DotnetFirstTimeUseConfigurer + sentinel, dotnet build -> MSB4018 from NuGet.targets, same mkdtemp, in RestoreTask wrapped build + test -> Build succeeded; Passed! Failed: 0, Passed: 10 The middle row is what makes the constraint look gone: one wrapped run writes ~/.dotnet/<version>.dotnetFirstUseSentinel into the real home, outside the tmpfs, so a later unwrapped command clears the configurer and fails further in. The section now names that trap. Also corrects the socket finding: socket(AF_UNIX, …) is refused at creation with EPERM, not at bind, and socketpair() still succeeds — so only cross-process Unix-socket IPC is lost, which is why MSBuild's worker nodes are what breaks.
The review that dropped the wrapper reports clearing ~/.dotnet before its green unwrapped run, so the persisted sentinel doesn't account for it. Say what is reproducible — EROFS from a fresh sandbox, and the failure moving into RestoreTask once the sentinel exists — and leave the contrary result named and unexplained rather than attributed.
|
Reversed the wrapper removal in d0c1001, after re-running the constraint at this branch's head. From a fresh sandbox — $ dotnet build prqlc/bindings/dotnet -m:1
... mkdtemp("/tmp/.dotnet.i1m5Ci") == nullptr; errno == EROFS;
at Microsoft.DotNet.Configurer.DotnetFirstTimeUseConfigurer.Configure()
That doesn't reconcile with the green unwrapped run above, which reports clearing Corrected in the same push: |
prql-bot
left a comment
There was a problem hiding this comment.
The /tmp constraint is real — EROFS reproduces here from a fresh sandbox, so restoring the wrapper in ec3ef9f was the right call over 358ca86. But the wrapper isn't what the constraint needs, and the result this section records as unreconciled turns out to be explained.
The whole /tmp dependency is one named mutex. NuGet.Common.Migrations.MigrationRunner.Run opens a Mutex called NuGet-Migrations, backed by a hard-coded /tmp path — but only when the marker for its migration count is absent. That marker lives at ${XDG_DATA_HOME:-~/.local/share}/NuGet/Migrations/1, which is in neither ~/.dotnet nor ~/.nuget. Creating it removes the constraint outright: with /tmp still read-only and no wrapper, dotnet new, build and test all run green.
That also reconciles the contrary result. The earlier session cleared ~/.dotnet and ~/.nuget — the two directories this section still names — and the marker, sitting outside both, survived. Its green unwrapped run wasn't a misread; it was a sandbox that had already been through a wrapped dotnet command, which writes the marker into the real home.
So the wrapper buys nothing over a touch, and it isn't free: unshare --map-root-user --mount runs the build as uid 0 and relocates every absolute path in the output to $TMPDIR/with-writable-tmp-checkout/…. I've pushed the replacement — the marker moves to sandbox_setup in .config/tend.yaml, where this repo already handles "the sandbox is missing a precondition every session needs", so no session has to remember it. -m:1, the recipe and the NU1903 note stay; with-writable-tmp.sh goes.
This is not 358ca86 again. That commit dropped the wrapper on an unexplained green run and left the documented recipe failing for the next session. This one names the mechanism, replaces it, and runs the whole test-dotnet job to Passed! - Failed: 0, Passed: 10 without it.
One caveat now in the skill: 1 is NuGet's migration count as of SDK 10.0.400. A later SDK bumps it, the marker stops matching and the EROFS returns — self-announcing, but it needs the new number in .config/tend.yaml.
Controlled runs
All from this session, /tmp read-only throughout (touch /tmp/probe → Read-only file system before and after).
| sandbox state | command | result |
|---|---|---|
fresh: ~/.dotnet holds tools/ only, no ~/.nuget, no marker |
dotnet new console |
mkdtemp("/tmp/.dotnet.wZHyxR") == nullptr; errno == EROFS, in DotnetFirstTimeUseConfigurer |
after one wrapped dotnet build |
unwrapped dotnet build … -m:1 |
Build succeeded. |
marker deleted; sentinel and ~/.nuget kept |
unwrapped dotnet build … -m:1 |
MSB4018 from NuGet.targets(198,5), same mkdtemp, inside RestoreTask |
touch …/NuGet/Migrations/1, nothing else changed |
unwrapped dotnet build … -m:1 |
Build succeeded. |
marker only; ~/.dotnet sentinels deleted, ~/.nuget moved away |
unwrapped dotnet new console |
Restore succeeded. |
| marker only | unwrapped cargo build -p prqlc-c → copy libprqlc_c.* → dotnet test … -m:1 |
Passed! - Failed: 0, Passed: 10 |
Rows 3 and 4 are the pair that isolates it: deleting only the marker brings the failure back, and re-creating only the marker takes it away. Row 5 shows it covers the first-run configurer too, not just RestoreTask.
The section's description of how the failure moves is exactly right and worth keeping — once ~/.dotnet/<version>.dotnetFirstUseSentinel exists the EROFS surfaces as a generic MSB4018 rather than a bare IOException, so a probe matching on the original message misses it.
Two negatives, so they don't get re-derived: NUGET_MIGRATIONS_DISABLED=1 does not suppress the mutex (still EROFS), and the -m:1 finding reconfirms unchanged — unwrapped dotnet build without it gives Build FAILED. with 0 Warning(s), 0 Error(s) and exit 1.
with-writable-tmp.sh itself is sound — shellcheck-clean, correct argument passing and mount ordering, and it does what it claims. The objection is only that the job doesn't need it.
The read-only /tmp only blocks dotnet because NuGet.Common.Migrations.MigrationRunner opens a named Mutex backed by a hard-coded /tmp path, and it does that only when the marker for its migration count is absent. Creating that marker lets the whole test-dotnet job run unwrapped, so with-writable-tmp.sh and its uid-0 namespace and bind paths go. The marker lives outside ~/.dotnet and ~/.nuget, which is why clearing those two left an earlier session with a green unwrapped run it could not explain. sandbox_setup creates it, so no session has to know.
prql-bot
left a comment
There was a problem hiding this comment.
-m:1 isn't accepted by every dotnet command, and dotnet new — one of the four the .config/tend.yaml comment names — is where it breaks: dotnet new console -o a -m:1 exits 127 with '-m:1' is not a valid option. A session that follows Pass -m:1 to every dotnet command literally hits an unexplained failure in the one place this section exists to prevent one. build, test and restore all accept it, and dotnet new doesn't need it — its post-create restore is a single project. I'm pushing that narrowing to the branch.
The marker mechanism itself holds up from a sandbox the authoring session never touched, which is the check this branch's two reversals actually call for. This session started with no ~/.local/share/NuGet/Migrations, no ~/.nuget, ~/.dotnet holding tools/ only, /tmp read-only, and no dotnet command yet run. dotnet new console aborted with exactly the documented mkdtemp("/tmp/.dotnet.XXXXXX") == nullptr; errno == EROFS out of MigrationRunner.Run inside DotnetFirstTimeUseConfigurer; a bare mkdir -p plus touch of the marker — nothing else — took it away, and the documented recipe then ran end to end to Passed! - Failed: 0, Passed: 10 with /tmp still read-only.
Independent reproduction
Every row from this review session, /tmp read-only throughout (touch /tmp/probe-review → Read-only file system).
| state | command | result |
|---|---|---|
| fresh sandbox, no marker | dotnet new console -o app |
mkdtemp("/tmp/.dotnet.iqTQpg") == nullptr; errno == EROFS at MigrationRunner.Run ← DotnetFirstTimeUseConfigurer.Configure |
| marker created, nothing else | dotnet new console -o app |
Restore succeeded. |
| marker only | dotnet build prqlc/bindings/dotnet |
Build FAILED. / 0 Warning(s) / 0 Error(s) / exit 1 |
| marker only | dotnet build prqlc/bindings/dotnet -m:1 |
Build succeeded., plus the two NU1903 warnings |
| marker only | cargo build -p prqlc-c → cp → dotnet test prqlc/bindings/dotnet -m:1 |
Passed! - Failed: 0, Passed: 10 |
| scratch single-project app | dotnet build (no -m:1) |
Build succeeded. — confirms why a quick probe misses the node failure |
The AF_UNIX claim reproduces directly too: socket(AF_UNIX, SOCK_STREAM) raises PermissionError/EPERM while socketpair() and socket(AF_INET, SOCK_STREAM) both succeed, so the section's account of what is and isn't lost is exact.
All eight .github/workflows/tend-*.yaml files carry the two new sandbox_setup lines, matching the eight that declare sandbox_setup at all. The skill hunk lands ~50 lines below #6335's, so the two are independent.
Changes to
prqlc/bindings/dotnet/had to be pushed unverified and judged from CI, becausedotnetin a tend session fails in two ways that don't name their own cause. This makes the wholetest-dotnetjob runnable in-session: onesandbox_setupprecondition, and onerunning-tendsection for the part that can't be preinstalled.A NuGet mutex that needs a writable
/tmp.NuGet.Common.Migrations.MigrationRunneropens a namedMutexwhose backing directory is a hard-coded/tmppath that no environment variable moves, and the sandbox's/tmpis read-only — sodotnet build,restore,testandnewall abort withmkdtemp("/tmp/.dotnet.XXXXXX") == nullptr; errno == EROFS, whiledotnet --infoand--versionsucceed because neither reaches the runner. The runner skips the mutex entirely once the marker for its migration count exists, sosandbox_setupcreates${XDG_DATA_HOME:-$HOME/.local/share}/NuGet/Migrations/1and no session has to know any of this.1is current as of SDK 10.0.400; a later SDK bumps it and needs the new number in.config/tend.yaml.MSBuild's worker nodes.
socket(AF_UNIX, …)is refused sandbox-wide withEPERMat creation —socketpairstill works, so only cross-process Unix-socket IPC is lost. A multi-node build dies in the node handshake and printsBuild FAILED.with0 Error(s)and no diagnostic;-m:1keeps it in one process and makes real errors visible again. A single-project build succeeds either way, which is why a quick probe misses it. That can't be preinstalled, so the skill section carries it along with the end-to-end recipe.Verified from a fresh sandbox —
~/.dotnetholdingtools/only, no~/.nuget,/tmpread-only throughout: with the marker alone,dotnet new consolerestores,dotnet build … -m:1succeeds, anddotnet test … -m:1reportsPassed! - Failed: 0, Passed: 10. Deleting only the marker brings theEROFSstraight back; re-creating only the marker takes it away again.How this arrived at the marker, after two reversals
Earlier commits on this branch added (
78d862f), removed (358ca86) and restored (ec3ef9f) awith-writable-tmp.shwrapper that randotnetunderunshare --map-root-user --mountwith a tmpfs over/tmp. The wrapper worked, but it made the build run as uid 0 and relocated every absolute path in its output to$TMPDIR/with-writable-tmp-checkout/…, and it left one result unexplained: a session had reported a fully green unwrappedbuildandtestafter clearing~/.dotnetand~/.nuget.The marker explains it. It lives under
~/.local/share, in neither of those two directories, so clearing both left it in place — and that session had already run wrappeddotnetcommands, which write it into the real home. Nothing was misread; the wrapper was simply a heavier way to satisfy a precondition atouchsatisfies, and it's gone from this PR.The failure also moves rather than disappearing, which is worth knowing when diagnosing it: once
~/.dotnet/<version>.dotnetFirstUseSentinelexists, the samemkdtempsurfaces as a genericMSB4018fromNuGet.targetsunderRestoreTaskinstead of a bareIOExceptionfrom the first-run configurer. Matching on the original message misses that.NUGET_MIGRATIONS_DISABLED=1does not suppress the mutex.Newtonsoft.Json9.0.1 reachesPrqlCompiler.Teststransitively throughMicrosoft.NET.Test.Sdkand restore flags it as NU1903, high severity. That predates this PR; the new section notes it so a future session doesn't chase it, and fixing it is a separate concern.The
.github/workflows/tend-*.yamledits are the generated mirror of the newsandbox_setuplines — the same edit the nightlytend/update-workflowsregen would make, applied now so the config isn't inert until then.#6335 adds a section to the same skill file ~50 lines away, so the two merge cleanly in either order.
No changelog entry — this changes agent instructions and CI config, not anything user-facing.