Skip to content

Bound managed Computer Use cursor movement to 50 ms - #667

Merged
igorcosta merged 2 commits into
mainfrom
computer-use-fast
Oct 11, 2026
Merged

igorcosta merged 2 commits into
mainfrom
computer-use-fast

Conversation

@igorcosta

@igorcosta igorcosta commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Long-distance managed cursor movement currently takes about 2.66 seconds with the CLI-pinned 0.28.2 engine. This change initializes a 50 ms movement budget before each named or implicit session's first input. The actual CLI MCP client measured subsequent movements at 53.8–54.5 ms on macOS, with first actions including setup at 51.6–82.1 ms.

Use the advertised start_session.cursor_motion field on newer engines or one legacy motion call on 0.28.2. Preserve explicit modern settings and later manual changes. Failed/cancelled setup blocks pending input and remains retryable; cancelling a waiter leaves another caller's initialization intact. AUTOHAND_COMPUTER_USE_GLIDE_MS accepts 0–5000, with zero retaining the engine's speed-based timing.

The PR fresh-install smoke also exposed missing commit metadata in the CI compilation recipe. Embed AUTOHAND_BUILD_GIT_COMMIT using the same Bun environment prefix as release builds, so moving the binary outside the checkout preserves its version identity. Add a failing-first workflow regression and verify compiled version output outside Git.

Validation:

  • Tests were added first: the initial implementation failed 16 of 18 regression cases. Final focused cursor/MCP checks pass all 77 tests, including reconnect and cancellation during initialization.
  • Lint, type checking, and tsup/DTS builds pass.
  • Installed Autohand macOS host and engine: private named and implicit overlay sessions pass a 250 ms maximum movement budget. A disposable AppKit fixture independently verified exactly one native button click per completed model trial.
  • Existing Rust shared cursor timing tests pass on both tick paths. Native Windows, X11, and Wayland environments are unavailable in this local session; no native timing claim is made for them.
  • Local bun run proof finished: 11,708 passing tests, six failures, and six worker startup errors. The failed cases cover installer startup, autoresearch, terminal handoff, and composer history. The focused installer case passes on untouched current main. The separate terminal proof was interrupted after 22 reported failures during disk/resource pressure. This is not a passing local aggregate result.
  • The first PR-event fresh-installer smoke exposed missing compiled commit metadata. On final head 0cf1835f5a250f3749fbac555dd27f170248319d, all three unit shards, all four terminal shards, and macOS/Ubuntu/Windows build/fresh-install checks pass for both push and PR events. Full proof completed: all 12,070 unit tests pass; terminal tests report 277 passed, three failed, and five skipped. The failures are in agent-run inspector, extensions, and task-list settings (two wait deadlines and one malformed configuration JSON read). These suites use AUTOHAND_DISABLE_COMPUTER_USE=1, so the cursor initialization path is disabled. Focused reruns on PR head and on the built exact CI merge tree each pass all 13 tests in those three suites. The aggregate run remains failed evidence. Earlier probes with local Bun 1.2.22 hit startup limits. A new isolated build of the exact PR synthetic merge 07bf2930ad647387b42f6140ffb50aceb46ed845 using the verified project-pinned Bun 1.4.2 passes signature verification, outside-checkout version identity, fresh installation, and reinstallation. The actual compiled CLI with a disposable profile and scripted local model replies performs three overlay movements through the installed native host, confirms the default 50 ms setting, and exits cleanly. This is native engine/compiled CLI proof, not live model or hardware-click proof.
  • The final diff from native-tested cd8c0663 changes only CI and version-test files; executable cursor source bytes remain unchanged.

Ready for independent review with the aggregate failure explicitly accounted for above; the active Main ruleset requires one approval. No review requirement is bypassed. Overlay timing is distinct from complete action latency: the old macOS engine's native click still costs roughly 1.2 seconds. Upstream quiet-action waits and a Weka decision controller are separate follow-up work. No engine repin or component release is included. Separately, the verified exact-merge-tree build was installed locally at the existing CLI path with the previous binary retained for rollback; the normal profile, native host, and driver are unchanged. An installed-path overlay smoke confirms the default 50 ms setting and clean shutdown. Main has not been merged while independent review is required.

igorcosta and others added 2 commits October 11, 2026 23:24
Initialize a 50 ms movement budget for named and implicit managed sessions,
using the advertised start-session field or compatible legacy motion tool.
Preserve explicit settings, isolate cancellation between callers, and retry
failed setup before allowing input. Document the bounded environment override
and cover the protocol behavior against spawned legacy and modern fixtures.

Co-authored-by: Autohand Evolve <code-noreply@autohand.ai>
Co-authored-by: Autohand Evolve <code-noreply@autohand.ai>
@igorcosta igorcosta added the full-proof Run the complete lint, unit, build, and terminal proof on GitHub label Oct 11, 2026
@igorcosta
igorcosta marked this pull request as ready for review October 11, 2026 11:36
@igorcosta
igorcosta merged commit cfe012d into main Oct 11, 2026
23 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-proof Run the complete lint, unit, build, and terminal proof on GitHub

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant