Skip to content

Stop the same proof miss in Claude, Cursor, and Codex. - #1323

Merged
mergify[bot] merged 2 commits into
mainfrom
stack/EdbertChan/reflect/same-repro-20261010/stop-same-proof-miss-claude-cursor-codex--a24f6a11
Oct 12, 2026
Merged

mergify[bot] merged 2 commits into
mainfrom
stack/EdbertChan/reflect/same-repro-20261010/stop-same-proof-miss-claude-cursor-codex--a24f6a11

Conversation

@EdbertChan

@EdbertChan EdbertChan commented Oct 11, 2026 •

Copy link
Copy Markdown
Owner

Summary

The checker that stops a test from standing in for the real program only reached Claude.

Cursor and Codex now get that same stop, in the form each editor already uses. A Codex session's command rows and patch rows count as the program and the edit.

Review Claim

The same proof miss stops the turn in Claude, Cursor, and Codex.

Review Lane

behavior

Review Unit

engine-runtime

Safety Invariant

The gate blocks only when a product command already ran, a file was edited after it, and a test runner then ran without that same program and subcommand. An unreadable transcript does not block. A session that never ran a product command does not block. Claude stops with an error code. Cursor sends the block text back as the next turn. Codex refuses the turn.

Confirmed by the user on 2026-10-10.

Slice Rationale

This slice wires the checker into the two editors it did not reach, and reads those editors' command and patch rows. The written skill page stays out, so the review stays on one unit.

Non-goals

Does not install the checker into a running editor. Does not change the written skill page. Does not change the app people run.

Architecture

Before

graph TD
    A["finding"] --> B["Claude stops"]
    A --> C["Cursor and Codex end the turn"]
Loading

After

graph TD
    A["finding"] --> B["Claude stops with an error code"]
    A --> C["Cursor sends the block text back"]
    A --> D["Codex refuses the turn"]
Loading

Measured

Command: git grep -c test_the_same_finding_blocks_claude_cursor_and_codex 3199d8af -- engine/hooks/prove-it-ship-gate/tests/test_hooks.py

  • base: (no output, exit 1)
  • head: HEAD:engine/hooks/prove-it-ship-gate/tests/test_hooks.py:1

Command: CATSTACK_HOOK_TEST_MODEL=gpt-5.6-sol python3 -m unittest discover -s engine/hooks/prove-it-ship-gate/tests -q

  • base: Ran 38 tests in 0.516s then OK, exit 0
  • head: Ran 40 tests in 1.173s then OK, exit 0

Test Plan

Test Plan
  • python3 scripts/ci/check_hook_test_coverage.py engine/hooks/prove-it-ship-gate — check_hook_test_coverage: OK (1 hook(s) checked)
  • ruff check --select E9,F engine/hooks/prove-it-ship-gate/codex_stop_check.py engine/hooks/prove-it-ship-gate/cursor_stop_check.py engine/hooks/prove-it-ship-gate/detect.py engine/hooks/prove-it-ship-gate/tests/test_hooks.py — All checks passed!
  • CATSTACK_HOOK_TEST_MODEL=gpt-5.6-sol python3 -m unittest discover -s engine/hooks/prove-it-ship-gate/tests -q — Ran 40 tests in 1.173s then OK
  • Not run: installing the checker into the live editor. Blocker: that changes every later stop in this editor until the pull request is merged and the installer is run

Revert Plan

Revert Plan
  • Safe to revert? Yes
  • Revert command: git revert 13bdf811
  • Post-revert steps: None
  • Data migration? No

Note

Medium Risk
Changes stop-hook behavior and transcript parsing for multiple agent harnesses; mis-parsing could under- or over-block turns, though the hook is designed to fail open on read errors.

Overview
Extends prove-it-ship-gate so the same “same-repro” stop (product CLI ran, edit happened, test runner ran without rerunning that CLI) fires on Cursor and Codex, not only Claude.

Codex gets a new Stop hook (codex.hook.json, codex_stop_check.py) that returns a block decision via the shared runtime. Cursor’s stop entrypoint now passes the correct stop event name into run_hook. detect.py broadens transcript parsing: more shell/edit tool names, cmd as well as command, and top-level function_call rows (exec_command, apply_patch) so Codex sessions participate in the same-repro logic. README documents per-harness blocking (exit 2 / followup / block). Tests cover Codex-shaped transcripts and assert Claude, Cursor, and Codex all surface the same finding.

Reviewed by Cursor Bugbot for commit 4bf00fb. Bugbot is set up for automated code reviews on this repo. Configure here.

@EdbertChan

Copy link
Copy Markdown
Owner Author

This pull request is part of a Mergify stack:

# Pull Request Link
1 Stop a test from standing in for the real program #1322
2 Stop the same proof miss in Claude, Cursor, and Codex. #1323 👈

@cursor

cursor Bot commented Oct 11, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: cc45d722-fd09-4f81-aaac-78351b089d03)

@EdbertChan
EdbertChan changed the base branch from stack/EdbertChan/reflect/same-repro-20261010/block-test-runner-pass-stands-real-cmd--161a67e5 to main October 11, 2026 19:21
@mergify

mergify Bot commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Queued — the merge queue status continues in this comment ↓.

EdbertChan and others added 2 commits October 11, 2026 19:29
The checker only ran for Claude, so the other two editors could still end the turn.

Co-authored-by: Cursor <cursoragent@cursor.com>
Change-Id: Ia24f6a11f6337ccc4310004e7a922bde00f250ff
@EdbertChan
EdbertChan force-pushed the stack/EdbertChan/reflect/same-repro-20261010/stop-same-proof-miss-claude-cursor-codex--a24f6a11 branch from 13bdf81 to 4bf00fb Compare October 11, 2026 19:30
@cursor

cursor Bot commented Oct 11, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: b0c2bc1a-61f2-4721-ae5c-8a7656b3fc06)

@EdbertChan

Copy link
Copy Markdown
Owner Author

@Mergifyio queue

@mergify

mergify Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Merge Queue Status

This pull request spent 50 minutes 59 seconds in the queue, including 50 minutes 37 seconds running CI.

Required conditions to merge
  • check-success = lint
  • check-success = test
  • check-success = validate

@mergify mergify Bot added the queued label Oct 12, 2026
@mergify mergify Bot mentioned this pull request Oct 12, 2026
6 tasks done
@mergify
mergify Bot merged commit 2745ccd into main Oct 12, 2026
6 checks passed
@mergify mergify Bot removed the queued label Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant