Skip to content

Pass Claude's structured answer to the plan parser unchanged (#166) - #169

Draft
mchwang wants to merge 1 commit into
feat/6-plan-library-alignmentfrom
feat/166-claude-structured-parse
Draft

mchwang wants to merge 1 commit into
feat/6-plan-library-alignmentfrom
feat/166-claude-structured-parse

Conversation

@mchwang

@mchwang mchwang commented Oct 12, 2026

Copy link
Copy Markdown
Contributor

Closes #166.

Stacked on #168 (base feat/6-plan-library-alignment). #168 added the agent-adapter test that described the old re-encoding; this PR replaces it. Retarget to main after #168 merges.

Summary

parseClaudeOutput read Claude's envelope with JSON.parse and returned JSON.stringify(structured_output). That round trip resolved duplicate keys (last wins) and rounded inexact integers before parseV1 saw the text, which bypassed the v1 parsing contract (schema/versions/1/semantics.md, "Deterministic input parsing").

  • agents/adapters/claude.ts: after JSON.parse validates the envelope, topLevelMembers cuts the exact source text of each top-level member. Planning answers return the exact text of structured_output, so parseV1 refuses a duplicate key or an inexact integer the envelope holds.
  • A planning envelope that repeats a top-level key is refused as malformed. The check runs before classification, so a repeated is_error cannot turn a refusal into a provider failure.
  • Every scanner loop is bounded by the input length. Non-object envelopes never reach the scanner, and could not hang it if they did.
  • The 1 MiB parser limit now applies to the bytes of structured_output in the envelope, not to a re-encoding.
  • Unchanged: question (non-schema) answers, provider-failure handling for well-formed envelopes, the supervisor's decoded-size checks (the slice is never longer than the raw output).

Limit

The Claude API and CLI may resolve duplicates or number spellings before writing the envelope. That is not verified: the real-CLI probes need credentials and do not run in CI. The docs (docs/implementation/agent-isolation.md) and the code comment say so. This PR guarantees that codeboost itself no longer normalizes the answer.

The recorded planning fixtures need no re-recording: their staleness hash covers the recorded repository files and argv only, and their text remains valid validator input.

Validation

Base f9c2e6b11c884e6814790d7f666c4e2cce3a67be (#168 head, baseline), head 083d9162e7b62d2f36a113bb1fb576e9a1d72e3f (validated head).

Gate Result
Typecheck Pass
Full suite without Docker 2034/2034 on the final tree
Docker suites Not affected by this diff; not run locally. CI is the gate
CI Pending

The four core tests (exact text, duplicate key, inexact integer, repeated top-level key) fail against the base decoder. Deliberate breaks of escape skipping, string skipping in nested values, the repeat refusal, its ordering before classification, and the non-object guard each fail a test.

Load-related failure, separate from product results: one full run at load average ~150 had a timing failure in pre-merge.test.ts ("stops an active command check when the operation-wide deadline expires"), a file this PR does not touch. It passed 3/3 alone and in the full rerun.

Review record

Round Reviewer Pair Findings Changed / declined
1 Independent agent (20,000-case fuzz) f9c2e6b / uncommitted Repeated is_error classified before the repeat check; untested spacing, key spellings and raw-byte limit; assertion wording; doc precision; long lines Fixed
2 Independent agent (200,000-case fuzz) same Doc wrapping Fixed. Declined: the TypeError on a null envelope (unchanged from the base); question answers keeping last-wins (plain text, never parsed by parseV1)
3 Independent agent same Non-object guard untested and the scanner could hang without it; "exact text Claude wrote" overclaims what the CLI preserves Loops bounded by length; non-object tests added; claims narrowed to the envelope, and the API/CLI behaviour documented as unverified
4 Independent agent same Only inexact integers are refused, not any inexact number; one non-object case did not exercise the guard Fixed
5 Independent agent f9c2e6b / tree committed as 083d916 None Clean pass

Copilot review has not been requested: this PR is a draft and stacked.

Review-lesson audit

  • Overclaiming wording in docs and test titles: covered by CLAUDE.md "Treat every behavioural claim … as something to verify".
  • A loop that can hang only when a guard is removed: one-off defensive hardening of this scanner.
  • Unverifiable upstream behaviour: recorded as untested in the docs (CLAUDE.md, behavioural claims rule).
    No new AGENTS.md rule is needed.

🤖 Generated with Claude Code

parseClaudeOutput re-encoded structured_output with JSON.parse and
JSON.stringify, so a duplicate key or an inexact integer reached
parseV1 already resolved. Cut the exact text of structured_output from
the envelope with a length-bounded scanner instead, and refuse a
planning envelope that repeats a top-level key before classifying it.

Whether the Claude API or CLI resolves duplicates before writing the
envelope is not verified; the docs say so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant