fix(core): retry transiently busy e2fsprogs probes

[grs]

Summary

Prevent e2fsprogs preflight checks from failing when Linux temporarily reports a newly installed executable as busy.

Related Issue

Closes #4314

Changes

• Retry ExecutableFileBusy errors for up to one second when starting an e2fsprogs version probe.
• Keep cancellation responsive while waiting between retries.
• Continue to fail immediately for configuration, loader, and other execution errors.
• Add a Linux regression test that releases a write-open executable during the retry window.

Testing

• Checks appropriate to the affected code and behavior pass
• Unit tests added/updated (if applicable)
• E2E tests added/updated (if applicable)

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)
• Architecture docs updated (if applicable)

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[6639835]

Reviewed bounded executable-busy retry and cancellation/cleanup at 29c9c53. Only ExecutableFileBusy retries for at most twenty 50ms delays; other errors fail immediately. Each delay prioritizes cancellation, and success retains timeout/process-group cleanup/reap. Linux regression releases a writer after 100ms. No actionable issue found. Rust builds/Linux process tests were not run.

[github-actions]

Label test:e2e applied, but pull-request/4315 does not exist yet. A maintainer needs to comment /ok to test 29c9c53c22f505deca7478a7c06b9c73bfc9006c to mirror this PR. Once the mirror exists, re-apply the label or re-run Branch E2E Checks from the Actions tab.

[johntmyers]

/ok to test 29c9c53

[johntmyers]

gator-agent

PR Review Status

The bounded executable-busy retry has no blocking review findings. It retries only the transient Linux error, keeps cancellation responsive, and preserves process cleanup; the added Linux regression test covers recovery after the executable writer closes.

Branch Checks, Helm Lint, and the standard E2E suite are queued or running for the current head after applying test:e2e and posting /ok to test. Trivy Changes is still awaiting workflow approval, and sandbox policy prevents Gator from approving that run.

Action required: A maintainer must approve the current-head Trivy Changes run, or the operator must allow the scoped workflow-approval endpoint.

Blocking findings: None.
Carried findings: None.

Gator metadata

• Validation: Focused VM host-tool preflight bug fix with reproduction and acceptance criteria in #4314; no duplicate found.
• Docs: No update needed for an internal transient-error retry with unchanged commands, configuration, and diagnostics.
• Checks: DCO and vouch passed; Branch Checks and Helm Lint dispatched; Trivy Changes requires workflow approval.
• E2E: test:e2e applied; current-head Branch E2E Checks run 38296984577 dispatched; no GPU or Windows label needed.
• Head SHA: 29c9c53c22f505deca7478a7c06b9c73bfc9006c
• Base SHA: e406c2befbe65501409026d49259602669f7246a
• Merge base SHA: e406c2befbe65501409026d49259602669f7246a
• Patch ID: 1877b276d62dc6ae7682e56e98644d41b87fca0b
• Gator payload: 12
• Review mode: initial
• Previous reviewed SHA: none
• Review budget exhausted: no
• Maintainer decision required: no
• Next state: gator:blocked
• Blocked reason: workflow_approval_required

[johntmyers]

gator-agent

Monitoring Complete

Monitoring is complete because this PR has merged.

The remaining gator:blocked label is stale. I am removing it because there is nothing left for Gator to monitor on this PR.

Gator metadata

• Head SHA: 29c9c53c22f505deca7478a7c06b9c73bfc9006c
• Gator payload: 12
• Final status: merged