Repository navigation
Stop the same proof miss in Claude, Cursor, and Codex. - #1323
Merged
Conversation
Owner
Author
|
This pull request is part of a Mergify stack:
|
3 tasks done
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: cc45d722-fd09-4f81-aaac-78351b089d03) |
EdbertChan
changed the base branch from
stack/EdbertChan/reflect/same-repro-20261010/block-test-runner-pass-stands-real-cmd--161a67e5
to
main
October 11, 2026 19:21
Contributor
|
Queued — the merge queue status continues in this comment ↓. |
The checker only ran for Claude, so the other two editors could still end the turn. Co-authored-by: Cursor <cursoragent@cursor.com> Change-Id: Ia24f6a11f6337ccc4310004e7a922bde00f250ff
EdbertChan
force-pushed
the
stack/EdbertChan/reflect/same-repro-20261010/stop-same-proof-miss-claude-cursor-codex--a24f6a11
branch
from
October 11, 2026 19:30
13bdf81 to
4bf00fb
Compare
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: b0c2bc1a-61f2-4721-ae5c-8a7656b3fc06) |
Owner
Author
|
@Mergifyio queue |
Contributor
Merge Queue Status
This pull request spent 50 minutes 59 seconds in the queue, including 50 minutes 37 seconds running CI. Required conditions to merge
|
6 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The checker that stops a test from standing in for the real program only reached Claude.
Cursor and Codex now get that same stop, in the form each editor already uses. A Codex session's command rows and patch rows count as the program and the edit.
Review Claim
The same proof miss stops the turn in Claude, Cursor, and Codex.
Review Lane
behavior
Review Unit
engine-runtime
Safety Invariant
The gate blocks only when a product command already ran, a file was edited after it, and a test runner then ran without that same program and subcommand. An unreadable transcript does not block. A session that never ran a product command does not block. Claude stops with an error code. Cursor sends the block text back as the next turn. Codex refuses the turn.
Confirmed by the user on 2026-10-10.
Slice Rationale
This slice wires the checker into the two editors it did not reach, and reads those editors' command and patch rows. The written skill page stays out, so the review stays on one unit.
Non-goals
Does not install the checker into a running editor. Does not change the written skill page. Does not change the app people run.
Architecture
Before
graph TD A["finding"] --> B["Claude stops"] A --> C["Cursor and Codex end the turn"]After
graph TD A["finding"] --> B["Claude stops with an error code"] A --> C["Cursor sends the block text back"] A --> D["Codex refuses the turn"]Measured
Command:
git grep -c test_the_same_finding_blocks_claude_cursor_and_codex 3199d8af -- engine/hooks/prove-it-ship-gate/tests/test_hooks.py(no output, exit 1)HEAD:engine/hooks/prove-it-ship-gate/tests/test_hooks.py:1Command:
CATSTACK_HOOK_TEST_MODEL=gpt-5.6-sol python3 -m unittest discover -s engine/hooks/prove-it-ship-gate/tests -qRan 38 tests in 0.516sthenOK, exit 0Ran 40 tests in 1.173sthenOK, exit 0Test Plan
Test Plan
python3 scripts/ci/check_hook_test_coverage.py engine/hooks/prove-it-ship-gate—check_hook_test_coverage: OK (1 hook(s) checked)ruff check --select E9,F engine/hooks/prove-it-ship-gate/codex_stop_check.py engine/hooks/prove-it-ship-gate/cursor_stop_check.py engine/hooks/prove-it-ship-gate/detect.py engine/hooks/prove-it-ship-gate/tests/test_hooks.py—All checks passed!CATSTACK_HOOK_TEST_MODEL=gpt-5.6-sol python3 -m unittest discover -s engine/hooks/prove-it-ship-gate/tests -q—Ran 40 tests in 1.173sthenOKRevert Plan
Revert Plan
git revert 13bdf811Note
Medium Risk
Changes stop-hook behavior and transcript parsing for multiple agent harnesses; mis-parsing could under- or over-block turns, though the hook is designed to fail open on read errors.
Overview
Extends prove-it-ship-gate so the same “same-repro” stop (product CLI ran, edit happened, test runner ran without rerunning that CLI) fires on Cursor and Codex, not only Claude.
Codex gets a new Stop hook (
codex.hook.json,codex_stop_check.py) that returns a block decision via the shared runtime. Cursor’s stop entrypoint now passes the correct stop event name intorun_hook.detect.pybroadens transcript parsing: more shell/edit tool names,cmdas well ascommand, and top-levelfunction_callrows (exec_command,apply_patch) so Codex sessions participate in the same-repro logic. README documents per-harness blocking (exit 2 / followup / block). Tests cover Codex-shaped transcripts and assert Claude, Cursor, and Codex all surface the same finding.Reviewed by Cursor Bugbot for commit 4bf00fb. Bugbot is set up for automated code reviews on this repo. Configure here.