Skip to content

Flag absence and capability claims as unverified-shaped in diu-stop - #1383

Closed
EdbertChan wants to merge 16 commits into
stack/EdbertChan/reflect/prove-it-hooks-20261009/list-run-check-first-unverified-claim-notice-stop--7b3f426ffrom
stack/EdbertChan/reflect/prove-it-hooks-20261009/flag-absence-capability-claims-unverified-shaped--6f50c350
Closed

EdbertChan wants to merge 16 commits into
stack/EdbertChan/reflect/prove-it-hooks-20261009/list-run-check-first-unverified-claim-notice-stop--7b3f426ffrom
stack/EdbertChan/reflect/prove-it-hooks-20261009/flag-absence-capability-claims-unverified-shaped--6f50c350

Conversation

@EdbertChan

@EdbertChan EdbertChan commented Oct 13, 2026 •

Copy link
Copy Markdown
Owner

Summary

A checker reads each reply before Claude sends it. It asks for proof whenever a claim sounds like a fact about the world.

The problem: claims like "there is no such call" or "that can't be done" slipped through with no proof, though they are claims too.

The fix: those phrases now count. A reply with one needs real output or an honest tag, like any other claim.

Replies without such phrases stay silent.

Review Claim

Claims that something does not exist or cannot be done now need proof or a tag before the reply is sent.

Review Lane

behavior

Review Unit

engine-runtime

Safety Invariant

Each change only affects when a checker warns about or blocks a reply. Checkers never edit files or run commands, and each change adds tests that fail before and pass after.

Slice Rationale

This slice stands alone: it only widens which phrases trigger the notice. The wording and retry changes sit in the previous slice, and the citation cleanup in the next one.

Non-goals

  • Does not change the text of the notice.
  • Does not change the retry or release rules.
  • Does not cover phrases beyond the five absence and capability forms in the diff.

Measured

Command: source scripts/test/hermetic_env.sh; python3 engine/hooks/diu-stop/tests/test_hooks.py with this commit's fixtures and the parent's checker code (base) versus this commit's checker code (head).

  • base: diu-stop: Ran 98 tests, FAILED (failures=5)
  • head: diu-stop: Ran 98 tests, OK

Test Plan

Test Plan
  • source scripts/test/hermetic_env.sh; python3 engine/hooks/diu-stop/tests/test_hooks.py -> Ran 98 tests, OK
  • python3 scripts/ci/check_hook_test_coverage.py engine/hooks/diu-stop -> check_hook_test_coverage: OK (1 hook(s) checked)
  • A fixture that fires (absence claims) and a fixture that stays silent (plain replies) are added.

Revert Plan

Revert Plan
  • Safe to revert? Yes
  • Revert command: git revert <merge commit of this PR>
  • Post-revert steps: None
  • Data migration? No

🤖 Generated with Claude Code


Note

Medium Risk
Stop-hook behavior changes can block more assistant turns; the doubled Mergify checks timeout affects how long queued PRs wait before dequeue.

Overview
Tightens agent proof gates and merge-queue ops, alongside new CI timing tooling and cat-mode guidance.

Hooks: diu-stop now treats absence/capability phrases (“has no”, “there is no”, “can’t be done”, etc.) as unverified-shaped claims and tells the model to run the check first, tag only as a last resort. unverified-tag-ledger blocks replies that pair a CAT-UNVERIFIED tag with a read-only handback (“say ‘test it’”), and limits how stop_hook_active bypasses the no-verification-tool block. frustration-watchdog treats “I’m running/checking … now” as ending an impatience wait.

Merge queue: admin-bypass checks_timeout goes from 60 to 120 minutes in .mergify.yml. land-stack adds verify_stack.py --queue-feedback to parse Mergify’s Merge Queue Status comment (timeout, conflict, checks-failed, in-queue, waiting) with tests on real comment fixtures.

Product: New ci-job-timing skill and ci_job_timing.py (Datadog [CI_STEP] logs + optional GitLab needs/gap context).

Corpus / cat-mode: Rules for local-only repos (no push/gh), cheaper data sources before capped reruns, measured duration answers, small-window proof before widening analytics paths, and a learned rule to list the user’s own authored items first when tracing ownership. pre-push stops using mapfile for zsh compatibility.

Reviewed by Cursor Bugbot for commit c00094b. Bugbot is set up for automated code reviews on this repo. Configure here.

@EdbertChan
EdbertChan added this pull request to stack #1385 October 13, 2026 04:39
@cursor

cursor Bot commented Oct 13, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: 33708426-d9d5-499c-b200-a5b35fd5b09e)

EdbertChan and others added 8 commits October 13, 2026 05:51
Change-Id: Id0dec9283a11dc8cf7089f236efd154b5d84b090

Co-authored-by: Edbert Chan <1916223+EdbertChan@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Change-Id: I23c2694dffd06040b854425181c3f91db98e3a83

Co-authored-by: Edbert Chan <1916223+EdbertChan@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Change-Id: Ic68644735337fb896acc5f3a44eb1815c5d57b60

Co-authored-by: Edbert Chan <1916223+EdbertChan@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
* invoker: wf-1791829352374-46/repair — Rebase PR #1309 onto main

Exit code: 0

* Fix cat-mode reference gate flag token

* invoker: wf-1791746481991-3/repair — Repair PR #1309 (failed check test)

Exit code: 0

* invoker: wf-1791822744313-30/repair — Repair PR #1310 (failed check validate)

Exit code: 0

* invoker: wf-1791850959113-81/repair — Rebase PR #1310 onto stack/EdbertChan/automate/dashboard-health-receipt/cat-mode-require-analytics-health-receipts--8efc336d

Exit code: 0

* invoker: wf-1791861093716-94/repair — Repair PR #1310 (failed check validate)

Exit code: 0

---------

Co-authored-by: CI Bot <ci@invoker.dev>
Change-Id: I038998b8c65e8267a8a5c96bf926270df24bca7e

Co-authored-by: Edbert Chan <1916223+EdbertChan@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
…est) (#1360)

Exit code: 0

Co-authored-by: CI Bot <ci@invoker.dev>
…mple phrasing. (#1311)

* invoker: wf-1791829352374-46/repair — Rebase PR #1309 onto main

Exit code: 0

* Fix cat-mode reference gate flag token

* invoker: wf-1791746481991-3/repair — Repair PR #1309 (failed check test)

Exit code: 0

* invoker: wf-1791822744313-30/repair — Repair PR #1310 (failed check validate)

Exit code: 0

* invoker: wf-1791850959113-81/repair — Rebase PR #1310 onto stack/EdbertChan/automate/dashboard-health-receipt/cat-mode-require-analytics-health-receipts--8efc336d

Exit code: 0

* cat-mode: lead analytics diagnosis rules with the general lesson.

Phrase-judge rejected leads that named only the trigger phrase; state the split and the stop-first catch as the lesson, with the triggers as examples.

Co-authored-by: Cursor <cursoragent@cursor.com>
Change-Id: I7dad9e54f9d280bb4d8f41ac06662293e39c9e05

* cat-mode: align publishing-analytics fixture asserts with revised example phrasing.

Co-authored-by: Cursor <cursoragent@cursor.com>
Change-Id: Idebc0f0ba74e47864bf479446a8e27b0574cdcde

* invoker: wf-1791828161279-44/repair — Repair PR #1311 (failed check test)

Exit code: 0

* invoker: wf-1791860491047-92/repair — Rebase PR #1311 onto stack/EdbertChan/automate/dashboard-health-receipt/cat-mode-lead-analytics-diagnosis-rules-general--7dad9e54

Exit code: 0

---------

Co-authored-by: CI Bot <ci@invoker.dev>
Co-authored-by: Cursor <cursoragent@cursor.com>
…lesson. (#1312)

* cat-mode: require analytics health receipts before dashboard narratives.

Agents kept narrating by-model health and patching charts before splitting volume, fill, resolve, and filter — lock that check into standing defaults.

Co-authored-by: Cursor <cursoragent@cursor.com>
Change-Id: I8efc336daf9878751865cf3c4bac4d55cd50733f

* cat-mode: lead analytics diagnosis rules with the general lesson.

Phrase-judge rejected leads that named only the trigger phrase; state the split and the stop-first catch as the lesson, with the triggers as examples.

Co-authored-by: Cursor <cursoragent@cursor.com>
Change-Id: I7dad9e54f9d280bb4d8f41ac06662293e39c9e05

* cat-mode: align publishing-analytics fixture asserts with revised example phrasing.

Co-authored-by: Cursor <cursoragent@cursor.com>
Change-Id: Idebc0f0ba74e47864bf479446a8e27b0574cdcde

* cat-mode: lead the multi-emitter backfill done-gate with the general lesson.

Co-authored-by: Cursor <cursoragent@cursor.com>
Change-Id: Iaa84683fcf5a66e8f70a0dbf99ebcf19acc1d7a7

* invoker: wf-1791863863942-101/repair — Repair PR #1312 (failed check validate)

Exit code: 0

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: CI Bot <ci@invoker.dev>
EdbertChan and others added 8 commits October 13, 2026 03:15
The pre-push review-unit check used a bash 4 builtin that the default macOS
shell (3.2) does not have, so every push from a Mac stopped at the hook. Read
the NUL-separated file lists with a portable loop instead.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

Change-Id: I186973c821653410e5d48a13d1acd2f2e4bb8433

Co-authored-by: Edbert Chan <1916223+EdbertChan@users.noreply.github.com>
…ho made their thing (#1387)

Asked which merge request produced a page the user owns, the assistant ran keyword
searches, hit timeouts and unrelated results, and reported none found. Listing the
user's own merge requests found it at once. One working-style rule line.


Change-Id: Icf84fb21574c8af5632bafe3e0b894131970e558

Co-authored-by: Edbert Chan <1916223+EdbertChan@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
…all-window first (#1386)

* cat-mode: look for a cheaper source, answer from measured numbers, small-window first

Adds three rules to the personal mode skill: look for a cheaper source before
running or repeating a slow capped bulk job, answer duration and window
questions from a measurement taken in the same turn, and prove a new data path
on a small window before widening it (folded into the existing analytics-tier
rule instead of a new bullet). Adds a fires and a stays-silent example for each
rule and tests that the skill and its references carry them.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

Change-Id: I875935c7afac476615cafaa68259595a7ee9f979

* invoker: wf-1791867242871-105/repair — Repair PR #1386: failed_checks: validate;

Exit code: 0

---------

Co-authored-by: Edbert Chan <1916223+EdbertChan@users.noreply.github.com>
Co-authored-by: CI Bot <ci@invoker.dev>
* cat-mode: require analytics health receipts before dashboard narratives.

Agents kept narrating by-model health and patching charts before splitting volume, fill, resolve, and filter — lock that check into standing defaults.

Co-authored-by: Cursor <cursoragent@cursor.com>
Change-Id: I8efc336daf9878751865cf3c4bac4d55cd50733f

* cat-mode: lead analytics diagnosis rules with the general lesson.

Phrase-judge rejected leads that named only the trigger phrase; state the split and the stop-first catch as the lesson, with the triggers as examples.

Co-authored-by: Cursor <cursoragent@cursor.com>
Change-Id: I7dad9e54f9d280bb4d8f41ac06662293e39c9e05

* cat-mode: align publishing-analytics fixture asserts with revised example phrasing.

Co-authored-by: Cursor <cursoragent@cursor.com>
Change-Id: Idebc0f0ba74e47864bf479446a8e27b0574cdcde

* cat-mode: lead the multi-emitter backfill done-gate with the general lesson.

Co-authored-by: Cursor <cursoragent@cursor.com>
Change-Id: Iaa84683fcf5a66e8f70a0dbf99ebcf19acc1d7a7

* cat-mode: allowlist intervention-must-automate as a flag token.

The dashboard-health defaults name that audit flag in backticks; the
skill-existence check must treat it as a process token, not a skill path.

Co-authored-by: Cursor <cursoragent@cursor.com>
Change-Id: I1afa1d44a91d2886300ab5f17843b8e583dbda0c

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
…X' (#1381)

The ledger now refuses a reply that tags a claim and offers 'say ...' or
'tell me to test it' for a read-only check. The frustration watchdog accepts
'I am running X now' as a valid next step.


Change-Id: I3b5ef1297fe1f20eec746fd4f74ccc44c54516ca

Co-authored-by: Edbert Chan <1916223+EdbertChan@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
…asing the ledger block on a rewrite (#1382)

diu-stop now tells the agent to run the check now and tag only a check that
was attempted. The ledger re-blocks a tag with no check once after another
Stop hook fired, instead of releasing it at once.


Change-Id: I7b3f426f27a2983b458ac4eccd8c993d99476e82

Co-authored-by: Edbert Chan <1916223+EdbertChan@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
* cat-mode: local-only repos end in a local merge, never GitHub

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Change-Id: I26c788b179793511a9191bede9861bd8521bc347

* cat-mode: avoid backticked gh reference

* invoker: wf-1791870257543-107/repair — Repair PR #1378: failed_checks: test;

Exit code: 0

---------

Co-authored-by: Edbert Chan <1916223+EdbertChan@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: CI Bot <ci@invoker.dev>
'has no', 'there is no', 'no single', 'no such call' and 'can't be done'
now trigger the unverified-claim check, so a no-such-call claim needs
output or a tag like any other claim.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Change-Id: I6f50c350c18dfe9c42a78f4d5d17b48b151af2b9
@EdbertChan
EdbertChan force-pushed the stack/EdbertChan/reflect/prove-it-hooks-20261009/flag-absence-capability-claims-unverified-shaped--6f50c350 branch from 9e9314b to c00094b Compare October 13, 2026 10:25
@cursor

cursor Bot commented Oct 13, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: b31808d0-44e2-4381-a3c0-3c35935c96ac)

@EdbertChan
EdbertChan deleted the branch stack/EdbertChan/reflect/prove-it-hooks-20261009/list-run-check-first-unverified-claim-notice-stop--7b3f426f October 13, 2026 10:27
@EdbertChan EdbertChan closed this Oct 13, 2026
@EdbertChan EdbertChan reopened this Oct 13, 2026
@cursor

cursor Bot commented Oct 13, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: d06257dd-7446-4b52-8e5a-84a9b42b4d31)

@EdbertChan

Copy link
Copy Markdown
Owner Author

Landed on main as #1401 (same commit, rebased onto main). Closing this one; GitHub's stack refused to merge or retarget it.

@EdbertChan EdbertChan closed this Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant