Skip to content

Restore interaction mode after Computer Use tasks and streamline MCP responses - #669

Draft
igorcosta wants to merge 1 commit into
mainfrom
computer-use-flow
Draft

igorcosta wants to merge 1 commit into
mainfrom
computer-use-flow

Conversation

@igorcosta

@igorcosta igorcosta commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Computer Use currently asks for per-action approval unless users enable auto mode themselves, and a native action discovered after starting in plan mode is rejected before it can activate computer control. This change enters auto mode for the computer task and restores the previous mode after completion, failure, or cancellation. Later manual mode selections remain effective, including during provider recovery. Explicit restricted/dry-run settings, permission denials, hooks, and OS permission checks remain enforced.

Shared MCP stdio startup now waits for the child spawn event and protocol handshake instead of adding a fixed 100 ms timer. Newline framing decodes complete byte frames once, preserving UTF-8 split across pipe reads and avoiding repeated copies of large responses. A synthetic fragmented 2 MiB frame measured 34.23 ms before and 0.65 ms after; warm native MCP calls already measured about 1.1 ms, so this is not a claim of a comparable end-to-end computer-task speedup.

Closes #668.

Validation on the final source:

  • 206 focused unit tests passed; 102 real-subprocess MCP/command tests passed.
  • 12 built-terminal Computer Use tests passed without --yes for temporary-mode scenarios, including late native discovery from plan mode and cancellation restoration.
  • Full lint and typecheck passed. The exact 27 focused contracts and build steps passed on hosted macOS, Windows, and Linux at 423f8042784cb88ca8a8fd10aa858a32855bf5e3 in PR CI.
  • The compiled candidate and the actual installed local CLI passed overlay-only smoke checks through the existing native host, with six scripted-model requests and no approval overrides or hardware input. A fresh doctor reports MCP ready and both macOS permissions granted; account configuration, host, and driver hashes are unchanged.
  • All hosted unit shards (3), terminal shards (4), and three-OS build jobs passed at the exact candidate SHA in PR CI.
  • Required local bun run proof completed its unit stage with 12,061 passed, 22 failed, and 71 skipped tests (829 files passed, 12 failed, 5 skipped). It failed before the terminal stage. Failures include subprocess/PTY timeouts and other flows; the real MCP cases passed separately on both candidate and baseline. No claim is made that every local failure is pre-existing.
  • A separate full local built-terminal proof was interrupted after failures when all exact-SHA hosted unit and terminal shards had passed. Its aggregate result is incomplete; the focused Computer Use terminal suite passed all 12 cases. This PR stays draft for review of those local limits.

The installed native driver is unchanged. Native driver contract repair and desktop qualification are tracked independently in autohandai/computer-use#2; this CLI PR does not certify the native desktop matrix or add a Weka decision/action controller. Local installation is a candidate build with a retained rollback binary, not a published component release.

…responses

Enter temporary auto mode for admitted computer tasks, including native actions
discovered in plan mode, and restore prior state without overriding manual mode
choices. Preserve explicit restrictions, permission policies, and hook checks.

Remove the fixed stdio startup delay and decode complete newline-framed byte
messages to preserve fragmented Unicode and avoid repeated large-frame copies.
Cover the built terminal behavior and shared contracts in the existing OS jobs.

Addresses #668.

Co-authored-by: Autohand Evolve <code-noreply@autohand.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Computer Use tasks should enter temporary auto mode and avoid repeated orchestration delays

1 participant