Skip to content

Block a CAT-UNVERIFIED tag that hands a read-only check back as 'say X' - #1381

Merged
EdbertChan merged 1 commit into
mainfrom
stack/EdbertChan/reflect/prove-it-hooks-20261009/block-cat-unverified-tag-hands-read-only-check--3b5ef129
Oct 13, 2026
Merged

EdbertChan merged 1 commit into
mainfrom
stack/EdbertChan/reflect/prove-it-hooks-20261009/block-cat-unverified-tag-hands-read-only-check--3b5ef129

Conversation

@EdbertChan

@EdbertChan EdbertChan commented Oct 13, 2026 •

Copy link
Copy Markdown
Owner

Summary

Small checker scripts read each reply before Claude sends it. One checks that claims come with proof.

The problem: Claude could mark a claim "unverified" and then ask the user to say "test it" for a check Claude can run itself.

The fix: that reply is now refused and Claude is told to run the check. A second checker now accepts "I am running it now" as a valid next step.

Offers tied to real actions, like pushing or deleting, still pass.

Review Claim

A reply that hands a harmless check back to the user is refused, and a reply saying the check is running now is accepted.

Review Lane

behavior

Review Unit

engine-runtime

Safety Invariant

Each change only affects when a checker warns about or blocks a reply. Checkers never edit files or run commands, and each change adds tests that fail before and pass after.

Slice Rationale

This is the first slice: it adds a new refusal rule, and the later slices reuse the same ledger checker. It is one claim: stop handing checks back. The other slices change wording, retry handling, and citations.

Non-goals

  • Does not change when a tag is accepted after a real attempt.
  • Does not touch offers that name a write such as push, deploy, or delete.
  • Does not change the word-count or claim checks.

Measured

Command: source scripts/test/hermetic_env.sh; python3 engine/hooks/<hook>/tests/test_hooks.py with this commit's tests and the parent's checker code (base) versus this commit's checker code (head).

  • base: frustration-watchdog: Ran 25 tests, FAILED (failures=1); unverified-tag-ledger: Ran 57 tests, FAILED (failures=3)
  • head: frustration-watchdog: Ran 25 tests, OK; unverified-tag-ledger: Ran 57 tests, OK

Test Plan

Test Plan
  • source scripts/test/hermetic_env.sh; python3 engine/hooks/frustration-watchdog/tests/test_hooks.py -> Ran 25 tests, OK
  • source scripts/test/hermetic_env.sh; python3 engine/hooks/unverified-tag-ledger/tests/test_hooks.py -> Ran 57 tests, OK
  • python3 scripts/ci/check_hook_test_coverage.py engine/hooks/frustration-watchdog -> check_hook_test_coverage: OK (1 hook(s) checked)
  • python3 scripts/ci/check_hook_test_coverage.py engine/hooks/unverified-tag-ledger -> check_hook_test_coverage: OK (1 hook(s) checked)
  • Positive and negative cases are in the new tests: a "say test it" offer fires, an offer tied to a write stays silent.

Revert Plan

Revert Plan
  • Safe to revert? Yes
  • Revert command: git revert <merge commit of this PR>
  • Post-revert steps: None
  • Data migration? No

🤖 Generated with Claude Code

The ledger now refuses a reply that tags a claim and offers 'say ...' or
'tell me to test it' for a read-only check. The frustration watchdog accepts
'I am running X now' as a valid next step.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Change-Id: I3b5ef1297fe1f20eec746fd4f74ccc44c54516ca
@EdbertChan
EdbertChan added this pull request to stack #1385 October 13, 2026 04:39
@cursor

cursor Bot commented Oct 13, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: 47dcc35c-5560-404c-b2c0-6099c0cb0852)

@mergify

mergify Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Queued — the merge queue status continues in this comment ↓.

@EdbertChan

Copy link
Copy Markdown
Owner Author

@Mergifyio queue

@mergify

mergify Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Merge Queue Status

This pull request spent 4 minutes 22 seconds in the queue, with no time running CI.

Reason

Pull request #1381 has been merged manually at 26a533a

Hint

You were too fast!

Waiting for
  • check-success = test
All conditions
  • check-success = test
  • check-success = lint
  • check-success = validate

Tick the box to put this pull request back in the merge queue (same as @mergifyio queue).

  • Requeue this pull request

@mergify

mergify Bot commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

queue

☑️ The pull request has been merged manually

Details

The pull request has been merged manually at 26a533a

@mergify mergify Bot removed the queued label Oct 13, 2026
@EdbertChan
EdbertChan deleted the stack/EdbertChan/reflect/prove-it-hooks-20261009/block-cat-unverified-tag-hands-read-only-check--3b5ef129 branch October 13, 2026 10:27
@EdbertChan
EdbertChan restored the stack/EdbertChan/reflect/prove-it-hooks-20261009/block-cat-unverified-tag-hands-read-only-check--3b5ef129 branch October 13, 2026 10:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant