diff --git a/README.md b/README.md index 416858b..3c7024e 100644 --- a/README.md +++ b/README.md @@ -1,1027 +1,123 @@ -

- Pitot — Pitot reports. Your controller decides. -

+# Pitot -

- Keep your coding agent. Add the behavior it is missing. -

+Put your own program at a coding agent's action boundary. - -

- Pitot agent E2E -

- -

Every supervised adapter must pass a binary-observed prompt → real action boundary → projected Consumer → Controller allow/deny → tool-result loop on Ubuntu, macOS, and Windows.

- -

Supervised adapters: Claude · Cursor · Codex · Devin · DeepSeek Harness · GitHub Copilot CLI · Gemini · Kimi Code · OpenCode · Pi · Qwen Code

- -

Supervised runtime capabilities: action control · Consumer delivery · explicit request

- - -

- One language-neutral boundary for the coding agents your team already uses. -

- -Pitot puts your own code at a coding agent's wired action boundary, so it can -allow, deny, or observe a proposed action. The shipped Codex, Qwen, and DeepSeek -integrations supervise shell tools; they do not mediate every file edit or tool -call. Applications can also issue explicit requests for their own action kinds. - -Pitot reports what happened. Your code decides what it means. - -## See it work with Kimi - -The fastest way to understand Pitot is to watch one real command get allowed and -another get denied. This walkthrough is exactly what Pitot's automated test suite -exercises on every commit, so the behavior below is verified, not aspirational. - -**1. Scaffold a sample shell policy.** This writes a runnable Controller that -allows shell commands by default and denies any command containing the canary -string `PITOT_DENY_ME`: - -```bash -pitot init --template shell-policy --language go --dir ./kimi-policy -cd ./kimi-policy -``` - -**2. Check your Kimi host wiring** (Pitot does not edit your Kimi config for you): - -```bash -pitot doctor --host kimi -``` +Pitot connects an agent to two kinds of application code: -If the `PreToolUse` hook is missing, add it to `~/.kimi-code/config.toml`: +| Role | What it does | +| --- | --- | +| Consumer | Receives projected events for observation and reporting. | +| Controller | Receives a registered request and returns a correlated allow or deny decision. | -```toml -[[hooks]] -event = "PreToolUse" -matcher = "Bash" -command = "pitot hook kimi" -``` - -**3. Run Kimi behind the Controller.** `pitot dev` starts the runtime, launches -the agent you name after `--`, and prints each decision: - -```bash -pitot dev --host kimi -- kimi -p "Run: echo hello" -``` - -An ordinary command is allowed and runs. Now ask for the canary: - -```bash -pitot dev --host kimi -- kimi -p "Run: PITOT_DENY_ME=1 echo nope" -``` - -The Controller denies it. The denied command never executes, and the denial -reason — `Pitot sample policy blocked the PITOT_DENY_ME canary.` — is returned to -Kimi as the blocked tool result. The `shell-policy` sample is a demonstration -tripwire, not a general shell-security control; the point is that *your* code -made the decision. - -### Devin uses ACP - -Devin exposes the same Pitot control semantics through a stateful ACP boundary, -so it does not need lifecycle-hook files: - -```bash -pitot doctor --host devin -pitot dev --host devin -- devin -p "Run: echo hello" -``` - -To attach to an already-running Pitot runtime, use the explicit single-prompt -surface: - -```bash -pitot acp devin --runtime "$PITOT_RUNTIME" --prompt "Run: echo hello" -``` +The shipped Codex, Qwen and DeepSeek integrations supervise shell tools. They do +not mediate every file edit or tool call. Your application can also send explicit +requests for its own action kinds. The runtime delivers `action.requested` events; +it does not collect model token usage. -Pitot selects only ACP's one-shot `allow_once` or `reject_once` options. -Interactive, resume, and multi-turn Devin sessions are not supported in this -initial adapter. +Pitot reports. Your Controller decides. -## Why Pitot? - -Building above coding agents usually forces one of two expensive choices: - -1. Maintain separate hooks, payload decoders, response formats, version quirks, - and diagnostics for every host. -2. Own or fork an entire coding-agent runtime just to gain a dependable event - boundary. - -Pitot provides a third option: keep the coding agents your users already chose, -integrate with their host boundary once, and build only the product that -differentiates you. - -

- Coding-agent hosts connect through Pitot to passive consumers and one controller per action kind -

- -Pitot separates two capabilities that are easy to blur: - -| Role | Receives | Can reply? | Typical use | -|---|---|---:|---| -| **Consumer** | Projected events | No | Usage, memory, analytics, reports | -| **Controller** | Registered requests | Yes | Approval, verification, policy, workflow | - -Pitot does not interpret Consumer output as Controller responses. A Controller is -statically registered for one request kind and returns at most one response for -the pending action. - -## Two small programs - -

- An action auditor consumes events while an approval controller receives a request and returns a response -

- -### 1. Audit shell requests without recording commands - -Configure a Consumer with an `omit` content projection: - -```yaml -consumers: - - id: action-audit - command: ["python3", "./examples/action-audit.py"] - events: ["action.requested"] - projection: - content: omit -``` - -Pitot writes newline-delimited JSON to the program's standard input: - -```json -{"pitot_version":"1","type":"action.requested","host":{"name":"claude"},"action":{"id":"act_7f2","kind":"shell"},"content":{"mode":"omit"},"observation":{"source":"host_hook","fidelity":"direct"}} -``` - -The Consumer is ordinary Python—no Pitot SDK required: - -```python -import json -import sys - -for line in sys.stdin: - event = json.loads(line) - print(json.dumps({ - "host": event["host"]["name"], - "action_id": event["action"]["id"], - "kind": event["action"]["kind"], - }), file=sys.stderr) -``` - -The command is projected out before bytes enter the Consumer pipe. Consumer -failure cannot allow or deny the waiting host action. - -### 2. Let a skill request approval - -A coding-agent skill can make a synchronous request: + +[Agent checks on main](https://github.com/operatorstack/pitot/actions/workflows/pitot-e2e.yml?query=branch%3Amain) -```bash -pitot request release.approval --data '{"release":"v1.4.0"}' --runtime "$PITOT_RUNTIME" -``` +Every supervised adapter must pass a binary-observed prompt → real action boundary → projected Consumer → Controller allow/deny → tool-result loop on Ubuntu, macOS, and Windows. -Register one Controller for that request kind: +Supervised adapters: Claude · Cursor · Codex · Devin · DeepSeek Harness · GitHub Copilot CLI · Gemini · Kimi Code · OpenCode · Pi · Qwen Code -```yaml -controllers: - release.approval: - id: local-approval - command: ["./examples/local-approval"] - deadline_ms: 2000 - on_timeout: deny - on_unavailable: deny -``` +Supervised runtime capabilities: action control · Consumer delivery · explicit request + -The Controller receives: +## Try the supervisory-control example -```json -{"pitot_version":"1","type":"control.requested","kind":"release.approval","action_id":"act_7f2","data":{"release":"v1.4.0"}} -``` +[The local delivery example](examples/supervisory-control/README.md) connects +Codex to a Qwen or DeepSeek worker through one shared Python supervisor. A worker +can build an artifact, but release is allowed only for its current tested revision. +A failed or invalidated test changes what the worker may do next. -It checks its own source of truth and returns one correlated response: +The replay demonstration runs on macOS and Linux. It uses real CLI processes +with a loopback replay provider and fake +credentials. It makes no real model calls. Its README explains the state, +feedback, verification and limits, and gives the pinned tools and run commands. +For application checks without those CLIs or network: -```json -{"pitot_version":"1","type":"control.response","controller_id":"local-approval","action_id":"act_7f2","outcome":"allow","message":"v1.4.0 is approved for publication."} +```sh +python3 -m unittest discover -s examples/supervisory-control -v ``` -Pitot validates the controller identity, action ID, deadline, schema, and -single-response rule before carrying the answer back. Pitot does not know what -“approved” means; the Controller owns that definition. - -## Install - -**Nobody installs Pitot per machine — the repository pins it.** `pitot init` -writes two substrate files alongside your tenant fragments: +## Install and register a Controller -- `.pitot/version` — one committed semver line, the only version authority -- `.pitot/bin/pitot` (+ `pitot.ps1`) — a committed shim that reads the pin, - hydrates exactly that release into a per-user cache - (`~/.cache/pitot//`, sha256-verified against the published - `checksums.txt`), and execs it +Use a release binary, or build the CLI from source: -Fresh clones, CI, and cloud agents run `.pitot/bin/pitot` with zero setup — -the first invocation hydrates, every later one is a cache hit. There is no -fallback to whatever binary happens to be on PATH; a missing release with no -network fails closed with a named error, and `PITOT_NO_HYDRATE=1` makes -hydration cache-only. - -**Upgrades are a reviewed diff.** `pitot upgrade` verifies the new release, -re-checks every tenant fragment against it, and rewrites the one pin line — -nothing else. Commit that diff and every clone hydrates the new version on -its next invocation; roll back by reverting it. - -```bash -pitot upgrade --check # report pinned vs latest -pitot upgrade # hydrate, validate tenants, rewrite .pitot/version -``` - -For a global CLI convenience (running `pitot init` in new repos), grab a -release binary or build from source: - -```bash +```sh go install github.com/operatorstack/pitot/cmd/pitot@latest -``` - -Typed SDKs install from our own registry through the distribution front door -— never public npm/PyPI, pinned to the CLI's version: - -```bash -pitot install typescript # scoped .npmrc + @operatorstack/pitot@ -pitot install python # .pitot/registry + operatorstack-pitot== -``` - -(`install typescript` creates a minimal private `package.json` when the -directory has none — npm honors a project registry config only inside a -project.) - -Inspect the effective local boundary, pin, and cache state at any time: - -```bash -pitot doctor -``` - -## Quickstart - -**1. Scaffold a Controller.** `pitot init` writes a runnable project — source -and a package manifest — and registers it as one tenant fragment under -`.pitot/conf.d/`. It never overwrites existing files unless you pass `--force`. -Pick a starting template with `--template`: - -```bash -pitot init --template shell-policy --language go --dir kimi-policy -``` - -``` -Initialized go controller (shell-policy) in kimi-policy -Files written: .pitot/conf.d/kimi-policy.yaml, kimi-policy/go.mod, kimi-policy/main.go -Next: - 1. Configure a supported host hook (see: pitot doctor --host HOST). - 2. Run: pitot dev --host HOST -- AGENT [ARGS...] - example: pitot dev --host kimi -- kimi -p "" -``` - -Available templates are `shell-policy` (allow/deny shell commands), -`release-approval` and `blank-controller` (request/response controllers), and -`blank-consumer` (a passive event reader). Without `--template`, `pitot init` -detects the language from the current directory or prompts you to choose. The -four first-class languages (`python`, `typescript`, `go`, `rust`) each generate a -complete project that builds after installing dependencies. - -**2. Run your agent behind it.** `pitot dev` discovers every fragment under -`.pitot/conf.d/`, starts the runtime and the declared Controllers, waits until -the runtime is ready, then launches the agent you name after `--` with -`PITOT_RUNTIME` set so its host hook finds the runtime. It prints each decision -as the agent makes it: - -```bash -pitot dev --host kimi -- kimi -p "Run: PITOT_DENY_ME=1 echo nope" -``` - -``` -Starting Pitot dev environment for host kimi... -Runtime ready. Starting agent: kimi -p Run: PITOT_DENY_ME=1 echo nope -Decisions: - [DENY] shell (act_1a9) — Pitot sample policy blocked the PITOT_DENY_ME canary. -Agent finished. Runtime stopped. -``` - -`--host` must name a supported agent (`claude`, `codex`, `copilot`, `cursor`, -`devin`, `gemini`, `kimi`, `opencode`, `pi`, `qwen`). Hook-based hosts must -already be wired to `pitot hook HOST` (see **Connect your agent** and -`pitot doctor --host HOST`); Devin connects over ACP and needs no hook file. -The runtime descriptor lives in a per-invocation -temporary path and is removed on exit, so concurrent `pitot dev` sessions never -collide. - -On Windows, `pitot dev` recognizes standard npm Node launchers for Codex and -Qwen, preserving their package entrypoint and arguments without invoking a -command shell. Custom `.cmd` wrappers are rejected with guidance to use -`-- node ABSOLUTE_ENTRYPOINT ARGS`. Prefer `-- COMMAND ARGS` for paths or -arguments containing spaces; `--exec` splits its value on whitespace. -When output is captured, an agent descendant that keeps an output pipe open -can cause an incomplete-output error after a two-second drain limit. This -releases the dev runtime; it does not terminate arbitrary descendant processes. - -**3. Swap the agent.** The same project — the same Controller and the same -fragment — works with any other supported host whose hook is wired. Change only -`--host` and the agent command: - -```bash -pitot dev --host cursor -- cursor-agent -p "Run: PITOT_DENY_ME=1 echo nope" -``` - -The boundary is language- and agent-neutral: one Controller, every agent. - -## Multiple tools, one Pitot - -Configuration is tenant-partitioned: every tool or person that registers -processes with Pitot owns exactly one fragment in `.pitot/conf.d/`, and no -tenant ever edits another tenant's file. The effective configuration is the -deterministic merge of all fragments in filename order: - -``` -.pitot/ - conf.d/ - boatstack.yaml # a tool's controller, written by its installer - interlock.yaml # another tool's controller, different request kind - my-policy.yaml # your own, scaffolded by `pitot init` -``` - -Each fragment is a complete, strictly parsed mini-config declaring -`controllers:` and/or `consumers:`. Merge rules: - -- **Consumers always compose.** Any number of tenants can observe - `action.requested` events. -- **A request kind has one owner.** Two fragments claiming the same kind (for - example `shell`) fail discovery with an error naming both files — a loud, - attributable conflict instead of two tools silently fighting over one - blocking hook. Controller and consumer ids must also be unique across - fragments. -- **`requires_protocol: "1"`** optionally pins the protocol version a fragment - was written against; a fragment this binary cannot honor fails discovery. -- **`dir:`** sets a process's working directory (relative to the repository - root), so each tenant's command stays project-relative: - -```yaml -controllers: - shell: - id: local-shell-policy - command: ["go", "run", "main.go"] - dir: "kimi-policy" - deadline_ms: 2000 - on_timeout: deny - on_unavailable: deny -``` - -Installing a second Pitot-based tool is therefore additive by construction: it -drops its own fragment next to yours, `pitot run`/`pitot dev` merge them, and -uninstalling it is deleting its fragment. - -## Advanced: manual runtime - -`pitot dev` is the recommended path. If you need to manage the runtime yourself -(for example, sharing one runtime across several long-lived agent sessions), -start it with repository-owned configuration and an owner-only runtime -descriptor: - -```bash -export PITOT_RUNTIME="${XDG_RUNTIME_DIR:-$TMPDIR}/pitot/project.json" -pitot run --runtime "$PITOT_RUNTIME" -``` - -With no `--config`, `pitot run` discovers and merges the repository's -`.pitot/conf.d/` fragments. Pass `--config PATH` to override discovery with one -explicit file (useful for tests and ad-hoc runtimes). - -One runtime owns each descriptor path. Pitot holds an operating-system lock on -the owner-only `PATH.lock` companion through startup, serving and cleanup. Leave -that empty file in place; the lock is released automatically on shutdown or a -process crash. Competing starters fail without changing the owner's descriptor. -An uncertain health result (including a timeout) also refuses takeover. A stale -descriptor is recovered only after a direct loopback connection refusal. -Older Pitot versions do not share this lock, so stop them before starting a new -runtime at the same path. - -Start coding-agent CLIs from the same environment. Their `pitot hook HOST` -commands discover the authenticated runtime through `PITOT_RUNTIME`. Without -that variable or `--runtime PATH`, hooks remain observation-only for backwards -compatibility. Once a runtime is explicitly selected, transport or -authentication failure blocks the controllable action. - -On Windows, set the descriptor in the launching PowerShell session: - -```powershell -$env:PITOT_RUNTIME = Join-Path $env:LOCALAPPDATA "Pitot\project.json" -pitot run --runtime $env:PITOT_RUNTIME -``` - -## Supported hosts - -Every host below normalizes its native blocking boundary to a `shell` action and -passes Pitot's language-neutral decoder conformance suite. The E2E column marks -adapters exercised by the cross-platform agent supervisor (the badge at the top); -the Platforms column names the operating systems each one is verified on. Kimi -additionally has an in-repo, no-model test that asserts the full allow **and** -deny control path end to end. - -| Host | Blocking boundary | Hook wiring | Platforms | Verified in this repo | -|---|---|---|---|---| -| Kimi Code | `PreToolUse` / Bash | native `config.toml` | Ubuntu · macOS · Windows | decoder + E2E + allow/deny control test | -| Claude | `PreToolUse` | native settings hook | Ubuntu · macOS · Windows | decoder + E2E | -| Cursor | `beforeShellExecution` | bridge (`integrations/cursor`) | Ubuntu · macOS · native Windows with PowerShell 7 | decoder + E2E | -| Codex | `PreToolUse` | bridge (`integrations/codex`) | Ubuntu · macOS · Windows | decoder + E2E | -| DeepSeek Harness | `tools/pre-execute` / `bash`, `pwsh` | native middleware ([setup](docs/deepseek-adapter.md)) | Ubuntu · macOS · Windows | decoder + E2E | -| Devin | `session/request_permission` (ACP) | stateful ACP transport | Ubuntu · macOS · Windows | decoder + E2E | -| GitHub Copilot CLI | `PreToolUse` | bridge (`integrations/copilot`) | Ubuntu · macOS · Windows | decoder + E2E | -| Gemini | `BeforeTool` | bridge (`integrations/gemini`) | Ubuntu · macOS · Windows | decoder + E2E | -| OpenCode | `PreToolUse` | bridge (`integrations/opencode`) | Ubuntu · macOS · Windows | decoder + E2E | -| Pi | `tool_call` | extension (`integrations/pi`) | Ubuntu · macOS · Windows | decoder + E2E | -| Qwen Code | `PreToolUse` | bridge (`integrations/qwen`) | Ubuntu · macOS · Windows | decoder + E2E | - -"Decoder" means Pitot correctly normalizes that host's payload into the stable -event envelope. It does not claim Pitot judges whether any command is safe — that -decision belongs to your Controller. Hosts run natively on Windows. Cursor's -Windows hook bridge requires PowerShell 7. - -## Host admission criteria - -A host earns a supervised adapter only when its blocking boundary can complete -Pitot's causal loop: a proposed command must reach a Controller, an allow or -deny decision must apply **before** the command runs, and — critically — a -denied action must return to the model so the agent can continue from the -blocked outcome rather than halting. This last requirement, deny-continuation, -is what distinguishes a supervisable boundary from one that can only abort. - -Devin is the worked example. Its lifecycle hooks (`PreToolUse`, -`PermissionRequest`) apply a denial but end the turn instead of handing the -rejected outcome back to the model, so they do not complete the loop in -non-interactive mode. Its Agent Client Protocol surface does: the client -selects `reject_once`, the canary never executes, and Devin makes a follow-up -model request carrying the rejection and continues. Pitot therefore ships Devin -over ACP and does not ship the hook wiring. The full investigation, including -the content-safe evidence receipt, is in -[docs/devin-adapter-research.md](docs/devin-adapter-research.md). - -## Connect your agent - -The per-host hooks below wire each agent's native blocking boundary to Pitot. -Hosts whose hook config lives **in the repository** are wired by Pitot itself: - -```bash -pitot init --host claude # also: cursor, codex, gemini -``` - -That writes exactly one marker-owned entry into the host's repo config -(`.claude/settings.json`, `.cursor/hooks.json`, `.codex/hooks.json`, -`.gemini/settings.json`), pointing at the repo shim — foreign entries are -never touched, and the committed fragment `.pitot/hooks/.fragment.json` -witnesses what was installed. `pitot doctor --host HOST` reports the entry as -FOUND, MISSING, or DRIFTED against that witness; `pitot doctor --host HOST ---fix` restores a drifted entry (Pitot's own entries only — that is the single -mutation doctor ever performs, and only on request). - -For Codex, Qwen and DeepSeek, doctor also checks the executable and the -integration's static setup on macOS, Linux and Windows. It reports independent -problems together. A `static setup: PASS` is scoped to the inspected files and -dependencies; `live enforcement: UNVERIFIED` means no agent action or Controller -decision was exercised. Codex session trust and normalized hook hashes still -require inspection in `/hooks`; stored trust that doctor cannot compare returns -a nonzero result. Unsupported command forms, conflicting Qwen hook definitions -and additional Codex configuration layers also require native inspection. - -With `PITOT_RUNTIME` unset, hooks remain observe-only. With it set, doctor checks -the selected runtime's identity through its local health endpoint. It does not -follow redirects, use an HTTP proxy, contact remote endpoints or submit a test -action. A healthy runtime does not establish that a Controller is available. -Plain doctor does not modify settings or grant trust; `--fix` performs the -supported wiring step, then runs the diagnostic checks separately. - -Hosts configured at **user level** (Kimi, Copilot, Qwen) or via **plugin -files** (OpenCode, Pi) keep the one-time manual edit below — Pitot does not -edit files outside the repository; `pitot init --host kimi` prints the exact -snippet. Once wired, both `pitot dev` and the manual runtime flow use the same -hook. - -### Claude Code / Codex - -```bash -pitot init --host claude -pitot init --host codex -``` - -Both wire a `PreToolUse` hook (matcher `Bash`) whose command runs the repo -shim: `"$CLAUDE_PROJECT_DIR"/.pitot/bin/pitot hook claude` (Codex uses the -repo-relative equivalent plus a PowerShell `commandWindows`). Exit `0` allows; -exit `2` blocks with the Controller's reason. - -For Codex, the project-local `.codex/` configuration must be trusted. Open -`/hooks` in the Codex CLI to review and trust the exact hook definition. New or -changed non-managed hooks are skipped until trusted; writing the hook file -alone does not activate supervision. See [Codex hook trust](https://learn.chatgpt.com/docs/hooks#review-and-trust-hooks). - -### Kimi Code - -Install Kimi Code on macOS or Linux using its official installer: - -```bash -curl -fsSL https://code.kimi.com/kimi-code/install.sh | bash -kimi --version -``` - -On Windows, use the official PowerShell installer: - -```powershell -irm https://code.kimi.com/kimi-code/install.ps1 | iex -kimi --version -``` - -Connect Kimi Code's blocking shell boundary to Pitot in -`~/.kimi-code/config.toml`: - -```toml -[[hooks]] -event = "PreToolUse" -matcher = "Bash" -command = "pitot hook kimi" -``` - -Kimi sends the hook payload to Pitot on standard input. Pitot exits `0` when -the request is accepted and `2` when input is malformed or the configured -Controller denies the action. Check -the non-interactive Kimi CLI after configuration with: - -```bash -kimi -p "Show the repository status" -``` - -See the [Kimi Code documentation](https://www.kimi.com/code/docs/en/) for CLI -authentication, configuration, and hook behavior. - -### GitHub Copilot CLI - -Copy `integrations/copilot/PreToolUse` to a stable executable path (or use -`PreToolUse.ps1` on Windows), then add the following Claude-compatible hook to -`~/.copilot/settings.json`: - -```json -{ - "hooks": { - "PreToolUse": [{ - "matcher": "Bash", - "hooks": [{"type": "command", "command": "/path/to/PreToolUse"}] - }] - } -} -``` - -The bridge keeps the blocking payload on Pitot's standard `hook_event_name` -and `tool_input.command` boundary and returns Copilot's structured native deny -reason when a Controller rejects the command. See the official -[Copilot CLI hooks reference](https://docs.github.com/en/copilot/reference/hooks-reference). - -### Cursor - -```bash -pitot init --host cursor -``` - -Writes the committed bridge `.pitot/bin/hooks/cursor-beforeShellExecution` -(which execs the repo shim and returns Cursor's native `permission: "deny"` -envelope, including the Controller message) and points `.cursor/hooks.json` at -it with `failClosed: true`. See Cursor's -[hooks documentation](https://cursor.com/docs/agent/hooks). - -The generated Cursor bridge above uses a Unix shell. For native Windows, use -`integrations/cursor/beforeShellExecution.ps1` with PowerShell 7 and keep -`failClosed: true` in the hook configuration. Put the checked PowerShell 7 -directory on Cursor's process `PATH` and invoke that same `pwsh.exe` explicitly -in the hook command. Cursor uses PowerShell to read and pass the hook payload; -Windows PowerShell 5.1 can corrupt Unicode before the bridge receives it and is -not supported by this bridge. The verification harness configures both shell -invocations without changing the user's `PATH` or installing WSL. - -### Gemini - -```bash -pitot init --host gemini -``` - -Writes the committed bridge `.pitot/bin/hooks/gemini-BeforeTool` and registers -it as a `BeforeTool` command hook for `run_shell_command`. The bridge -translates Pitot rejection into Gemini's structured `decision: "deny"` and -`reason` response so the model receives the blocked tool result. See the -[Gemini CLI hooks reference](https://geminicli.com/docs/hooks/reference/). - -### Qwen Code - -Copy `integrations/qwen/PreToolUse` to a stable executable path (or use the -Node-based `PreToolUse.cjs` bridge on Windows), then add this command hook to -`~/.qwen/settings.json`: - -```json -{ - "hooks": { - "PreToolUse": [{ - "matcher": "^run_shell_command$", - "hooks": [{"type": "command", "command": "/path/to/PreToolUse"}] - }] - } -} -``` - -The matcher uses Qwen's runtime shell tool ID. Compatibility aliases such as -`Bash` match exactly; anchoring that alias as `^Bash$` does not select the shell -tool. - -Qwen sends the native JSON payload on standard input. The bridge returns its -native structured allow or deny decision and preserves the Controller reason. -See the official -[Qwen Code hooks guide](https://qwenlm.github.io/qwen-code-docs/en/users/features/hooks/). - -### Pi - -Copy `integrations/pi/pitot.ts` into `~/.pi/agent/extensions/pitot.ts` (or the -repository-local `.pi/extensions/` directory). The extension converts Pi's -blocking `tool_call` event into Pitot's stable envelope and returns Pi's native -`block` response when Pitot rejects the request. See the official -[Pi extensions documentation](https://pi.dev/docs/latest/extensions). - -### Devin - -Devin needs no hook file. Pitot speaks to it over the Agent Client Protocol, -launching `devin acp` as a stdio JSON-RPC server and correlating each -`tool_call` command by its tool-call ID: - -```bash -pitot dev --host devin -- devin -p "Run: echo hello" -``` - -To attach to an already-running runtime, use the explicit single-prompt -surface: - -```bash -pitot acp devin --runtime "$PITOT_RUNTIME" --prompt "Run: echo hello" -``` - -Pitot maps an allow decision to ACP's one-shot `allow_once` and a deny to -`reject_once`; it never selects a persistent option such as `allow_always` or a -bypass mode. Attestation comes from the Controller receipts and canary rather -than a lifecycle-hook witness. This initial adapter is single-prompt; resume and -multi-turn Devin sessions are not yet supported. See -[docs/devin-adapter-research.md](docs/devin-adapter-research.md) for why ACP, -not hooks, is the supervised boundary. - -Pitot uses supervised local processes in v1. It starts declared Consumers and -Controllers itself, applies each projection before bytes enter the child pipe, -and exposes only a loopback endpoint authenticated by the owner-only runtime -descriptor. Pitot does not pass the capability token directly to child processes or include it -in its diagnostics. These are trusted programs running as the launching user: -they may read the owner-only descriptor and access its authenticated endpoint. -Pitot does not sandbox them or restrict their filesystem, environment, or network -access. Descriptor permissions exclude other users, not programs running as the owner. - -## Language-neutral by design - -Pitot's reference implementation is Go. Its public contract is not. - -Compatibility is defined by: - -- versioned JSON Schemas; -- newline-delimited JSON framing; -- explicit request/response state machines; -- capability and projection declarations; and -- language-neutral conformance fixtures. - -If a program can read JSON Lines from standard input, it can be a Consumer. If -it can return one schema-valid response on its dedicated output channel, it can -be a Controller. - -Official client libraries are optional conveniences—not a prerequisite and -not the source of protocol truth. - -## Event envelope - -Events identify their schema, host, source, and observation quality. -`session_id` is optional, opaque metadata supplied by the host; Pitot preserves -it without generating a substitute when it is unavailable. Group sessions by -`host.name` and `session_id`, rather than treating IDs from different hosts as -one namespace. The generated per-action ID correlates control replies and is -separate from the host's session identity. Session metadata is retained by every -content projection and does not authenticate a Controller. - -This example shows an envelope with session metadata: - -```json -{ - "pitot_version": "1", - "id": "evt_01J...", - "type": "action.requested", - "time": "2026-07-19T16:05:00Z", - "host": { - "name": "cursor", - "adapter_version": "1.0.0" - }, - "session_id": "sess_42", - "action": { - "id": "act_7f2", - "kind": "shell" - }, - "content": { - "mode": "sha256", - "sha256": "9f2c00a18f8164a5f13127c1c38e48e67d97db8a03ccdc0c7a1cb86842f90d85" - }, - "observation": { - "source": "host_hook", - "fidelity": "direct" - } -} -``` - -Adapters preserve host capability differences. A normalized field is never -presented as directly observed when the host omitted it or Pitot inferred it. - -## Runtime envelope admission - -The reference runtime validates event semantics before any Consumer delivery. -Events require version `1`, a nonempty host name, and an observation source of -`host_hook` or `host_event` with fidelity `direct` or `estimated`. Action events -also require a nonempty action ID and kind. Optional event metadata and -Controller-owned request data retain their existing meaning. - -Content representations are mutually exclusive: - -- `full` carries one JSON string containing the measured bytes, with no digest; -- `sha256` carries a 64-digit hexadecimal digest, with no full content; -- `omit` carries neither representation. - -Contradictory or malformed content is rejected before fanout, including events -submitted directly through the Go Manager. Projection validates content again -before using an already-projected representation. Validation errors contain no -payload or envelope identifiers. Valid envelopes retain the existing projection, -Controller correlation, timeout and unavailable-default behavior. - -The shared typed-envelope vectors are in -`conformance/fixtures/envelopes.jsonl`. This admission gate does not enable new -event delivery capabilities: runtime ingress still accepts only -`action.requested`. SDK readers apply the wire contract below before invoking handlers. - -## SDK wire contract - -The canonical v1 value constraints are published in [schema/v1.json](schema/v1.json). -Generated SDK types are structural conveniences; the Go, Python, TypeScript and -Rust runners validate raw input before constructing handler values. Optional -metadata may be absent, but explicit null is rejected for typed string/object -fields. Controller-owned `data` may contain any JSON value, including null. - -The JSON-Lines transport profile adds constraints that JSON Schema cannot check -after decoding: - -- UTF-8 and Unicode scalar strings, with unique decoded member names in every - object, including nested Controller data; -- one object per LF-delimited record, exact-case envelope fields, and no trailing - JSON or unknown envelope/typed-object fields; -- at most 4 MiB per record, excluding LF (CR counts), and 128 nested containers; -- token counts spelled as JSON integers, from 0 through 9007199254740991. Decimal - and exponent spellings are rejected even when their mathematical value is - integral; this preserves typed-reader compatibility and avoids silent rounding. - -Blank JSON whitespace lines are ignored. CRLF and a final nonempty record without -LF are accepted. Malformed and oversized lines are skipped with content-safe -diagnostics; oversized records are drained so the next valid record can proceed. -Controller results are validated before stdout, and absent response messages are -omitted. FIFO handler ordering is preserved, including asynchronous TypeScript. -Python admission does not depend on assertions and also runs under `python -O`. - -The shared raw corpus is [conformance/fixtures/wire.jsonl](conformance/fixtures/wire.jsonl). -Tests distinguish decoded JSON Schema constraints from transport constraints and -drive the real runners. Arbitrary Controller data still uses each language's -native JSON representation: numeric precision/range and absent/null distinctions -are not made identical by the SDKs. Generated constructors used directly do not -apply the runner admission gate. The Go API retains platform-sized `int` counters; -verification and shipped targets are 64-bit, while 32-bit callers have a narrower -native integer range. The runtime continues to deliver only -`action.requested` events; accepting `model.usage` in SDKs adds no host capability. - -## Controller guarantees - -For Go applications, `runtime.Start` validates the protocol and all process -registrations before launching any child. Empty embedded runtimes remain valid; -file configuration still requires at least one registration. An explicit request -can use any configured nonempty kind, and its unavailable/timeout defaults may -allow or deny. `DeliverControlledEvent` separately requires a live Controller -reply for an action event. - -Synchronous hooks are control channels: the host is blocked waiting for an -answer. Pitot makes that privilege explicit. - -- Exactly one Controller may register for a request kind. -- Registration is static, auditable configuration. -- The registration declares a deadline and unavailable/timeout defaults. -- The Controller deadline covers waiting for a busy Controller, writing its - request, and waiting for the response. A caller deadline can shorten that - budget; canceled requests do not wait indefinitely behind another request. -- If a write is interrupted, Pitot stops that Controller because its input may - contain a partial frame. Later requests use `on_unavailable` until the runtime - is restarted. A timeout after a complete write leaves the Controller running; - late replies cannot resolve a different action. -- Every response is bound to its pending action and Controller identity. -- Late, stale, duplicate, mismatched, and malformed responses are rejected. -- A pending action receives exactly one terminal resolution. -- Registration and its configuration fingerprint appear in diagnostics. - -The bridge mechanically enforces the declared default. It contains no approval, -safety, completion, or shipping policy engine. - -## Boundary faults - -Pitot distinguishes a broken measurement boundary from a judgment about the -work: - -```json -{ - "pitot_version": "1", - "type": "boundary.fault", - "host": "cursor", - "action_id": "act_7f2", - "reason": "empty-command" -} -``` - -Reason codes never contain prompts, commands, tool inputs, or outputs. A -Controller may choose to deny, retry, report, or escalate the fault according -to its own policy. - -## Privacy model - -Pitot is local and storage-free by default. - -- Raw host payloads terminate inside the adapter. -- Content projection is `full`, `sha256`, or `omit` per Consumer. -- Projection happens before delivery, not inside downstream applications. -- No network exporter is enabled implicitly. -- Pitot does not retain an event history unless a configured Consumer does. -- Consumer output is not wired to the Controller response channel. -- Pitot-generated event delivery and response rejection diagnostics omit envelope - identifiers and content. Configured process labels may appear in diagnostics. - Controller stderr and both Consumer output streams are forwarded to operator - stderr; this application-controlled output may contain sensitive information. - -Projection controls command content in each delivered event. It preserves other -metadata and does not prevent a trusted program from obtaining content elsewhere. -SHA-256 supports equality checks and comparison against guessed content; it is -not encryption. An existing digest can be narrowed to `omit`; requests that need -unavailable full content or a missing digest are rejected. Absent content stays -absent under every projection policy. - -Consumer delivery is bounded and best effort. Queue saturation and unavailable -processes produce diagnostics. An unexpected process exit or pipe-write failure -reports once that observation delivery may be incomplete, even without another -event. Queue acceptance and a successful pipe write are not acknowledgements that -an application processed the event. Pitot does not retry or retain dropped -observations; a Consumer can implement its own retention. Expected runtime -shutdown is quiet. Consumer pipe writes run separately from Controller resolution; -a slow operator diagnostic sink can still block existing delivery diagnostics. - -## What can you build? - -The reference runtime delivers `action.requested` to Consumers and routes -application-issued control requests to registered Controllers. The SDKs can -represent `model.usage`, but the shipped runtime does not collect or deliver -usage events. Token accounting and cross-agent memory need additional data -sources supplied by your application. - -- selective cross-agent memory; -- session and decision reports; -- token and cost attribution; -- reliability and host-compatibility diagnostics; -- OpenTelemetry exporters; -- human approval routers; -- security or compliance Controllers; -- delivery verifiers; -- custom agent interfaces over existing runtimes; and -- new Pitot-compatible coding-agent runtimes. - -## Threat model - -Pitot mediates the host's wired boundary and nothing else. Understanding where -that boundary ends is part of using it correctly. - -- **Only the wired boundary is mediated.** Pitot sees an action when the host - actually routes it through the configured hook or ACP transport. An action - the host takes through a path you did not wire is not observed. `pitot doctor - --host HOST` reports whether the boundary is present, and for repo-owned - hosts whether the entry has drifted. -- **User-level hook configs are user-editable.** For hosts wired at user level - (Kimi, Copilot, Qwen) the hook lives in a file the user owns and can change - or remove. Pitot does not police edits outside the repository; it reports the - current state through `pitot doctor`. -- **Without a runtime, hooks only observe.** A `pitot hook` invocation with no - `PITOT_RUNTIME` (or `--runtime`) selected records the action and exits - allowing it — observation-only, for backwards compatibility. It now prints a - one-line notice so this mode is never silent, and `pitot doctor` flags a host - that is wired but has no runtime. Once a runtime is explicitly selected, - transport or authentication failure blocks the controllable action rather - than falling open. -- **DeepSeek requires a live Controller reply.** Its native shell middleware - blocks execution without a selected runtime or an explicit matching reply, - even if ordinary runtime fallback defaults allow. This differs from the - observation-only behavior of command hooks with no runtime selected. -- **Hosts retain their own bypass options.** Some hosts expose persistent or - bypass permissions (for example Devin's ACP `allow_always` and switch-bypass - modes). Pitot never selects them — it uses only the one-shot `allow_once` / - `reject_once` options — but a human operating the host directly still can. -- **Kimi executes the action if its hook process crashes or times out.** This - is Kimi's host semantics, not a Pitot decision, and Pitot cannot override it: - a supervisory-control analysis of the Kimi lifecycle shows a reachable - transition from the pending state straight to an unsupervised execution when - the hook fails open, which no supervisor placed at the boundary can prevent. - `pitot doctor --host kimi` states this plainly. If your policy cannot - tolerate fail-open execution, prefer a fail-closed host (such as Cursor, - wired with `failClosed: true`) or Devin's ACP transport. -- **Codex hook failures can leave an action unsupervised.** A functioning - synchronous hook can block an action with a Controller's explicit denial or - Pitot's rejection of an unavailable selected runtime. Codex can continue - the tool when the hook cannot start, crashes, times out, or returns invalid - output. That host behavior is separate from a Controller's decision; a - passing allow/deny test does not guarantee that hook execution failures block - tools. Untrusted or changed hooks can also be skipped until reviewed. - -- **Qwen hook failures can leave an action unsupervised.** The pinned Qwen - release treats some hook startup errors, timeouts, and invalid output as - nonblocking. A functioning bridge can return a denial, but that does not - guarantee an action is blocked when the bridge itself fails. Valid structured - denial output and the host's blocking exit status are separate from these - nonblocking failure paths. - -None of these change the core contract: within the boundary Pitot mediates, -every pending action receives exactly one terminal resolution. **Pitot reports. -Your controller decides.** - -## What Pitot does not decide - -Pitot does not define whether: - -- work is correct or complete; -- an action is safe; -- a person granted approval; -- evidence satisfies a requirement; -- a claim is valid; or -- something may be shipped. - -Those meanings belong to your Controller. **Pitot reports. Your controller -decides.** - -## Project layout - -```text -pitot/ -├── schema/ public event and response schemas -├── protocol/ framing and state-machine specifications -├── adapters/ built-in coding-agent hook boundaries -├── sensor/ normalization and observation pipeline -├── bridge/ controller routing and response transport -├── projection/ full, sha256, and omit policies -├── config/ strict Consumer and Controller declarations -├── runtime/ authenticated ingress and local process delivery -├── conformance/ language-neutral fixtures and negative controls -├── examples/ local approval and Consumer examples -└── cmd/pitot/ reference Go executable -``` - -## Design principles - -1. **Report before interpretation.** Preserve what the host actually supplied. -2. **Make capability structural.** Consumer output is observation-only; only the - configured Controller response pipe participates in decisions. -3. **Keep host churn together.** Decoders and encoders share one compatibility - boundary. -4. **Project before delivery.** Privacy is enforced before content crosses the - process boundary. -5. **Correlate every answer.** A response can resolve only its pending action. -6. **Prefer one maintained implementation.** Use a language-neutral protocol - instead of rewriting the host boundary in every ecosystem. -7. **Extend standards where they fit.** Map to OpenTelemetry GenAI conventions - without erasing Pitot-specific provenance or semantic events. - -## Name - -A pitot probe measures pressure difference so another system can determine -airspeed. It does not fly the aircraft. - -Pitot applies the same separation to coding-agent tooling: measurement belongs -at the host boundary; interpretation and control belong downstream. - -## Contributing - -Start with the protocol and conformance fixtures. A new adapter should declare -its host capabilities, normalize supported events, classify boundary faults -without exposing content, encode Controller responses, and pass the shared -positive and negative fixture suite. It must also meet the host admission -criteria above — a boundary that can only abort on denial, rather than return -the outcome to the model, is not yet supervisable. - -See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup, the two adapter -transport classes (one-shot hook and stateful ACP-style transport), and the -cross-platform verification every adapter must pass. - -## License - -Apache-2.0 +pitot init --template shell-policy --language go --dir shell-policy +``` + +This creates a runnable Controller and registers it under `.pitot/conf.d/`. +The sample allows shell commands except those containing `PITOT_DENY_ME`. +It is a demonstration policy, not a general shell-security control. + +A released CLI also writes a repository version pin and launch shims during +`pitot init`. Commit those files so clones use the same release. Source builds +without an injected release version, including ordinary `go install` builds, +skip a missing pin and print a notice; their shims need a released version pin +before use. Cache-only and offline behavior is described in +[runtime setup](docs/runtime.md). + +## Connect Codex, Qwen or DeepSeek + +Host setup and a registered Controller are separate steps. + +| Host | Setup | What still needs checking | +| --- | --- | --- | +| Codex | `pitot init --host codex` | Review and trust the exact hook in Codex `/hooks`. Writing the file does not activate it. | +| Qwen | Configure the supplied Qwen bridge in the user hook settings. | Use its native `run_shell_command` matcher and inspect conflicting hooks. | +| DeepSeek | `pitot init --host deepseek` | Use the supported official Harness release and a shell Controller. | + +Run `pitot doctor --host codex`, `pitot doctor --host qwen`, or +`pitot doctor --host deepseek` to inspect setup. Doctor's static checks do not +prove live enforcement or grant hook trust. Plain doctor does not change settings; +`--fix` performs only supported wiring changes, then checks the result. + +[Host setup](docs/hosts.md) contains the commands, platform details and trust +requirements. [DeepSeek setup](docs/deepseek-adapter.md) describes its native +middleware and single-prompt interface. + +Once wired, `pitot dev --host HOST -- COMMAND ARGS` starts the declared runtime, +waits for readiness and launches the agent with its runtime descriptor. DeepSeek +also requires `--prompt`. Ordinary agent prompts can contact their configured +model provider; use the replay example above for a no-model-call demonstration. + +On Windows, standard npm Node launchers for Codex and Qwen are recognized without +batch evaluation; custom wrappers require an explicit Node entrypoint. The +explicit `-- COMMAND ARGS` form preserves argument boundaries. `--exec` splits +on whitespace. The generic subprocess launcher bounds captured-output waiting; DeepSeek and +Devin use separate ACP lifecycle handling. Pitot does not promise to terminate +arbitrary descendants. See [runtime setup](docs/runtime.md). + +## What the boundary guarantees + +A Controller response must match the pending action and registered Controller. +Consumers cannot resolve requests. Content is projected before delivery, using +`full`, `sha256` or `omit`. SHA-256 permits comparison against guessed content; +it does not conceal low-entropy secrets. + +Only the host's wired boundary is mediated. Codex and Qwen can continue an action +when a hook fails or is skipped. DeepSeek requires an explicit matching live +Controller reply for its native shell tools. These programs run as the current +user: Pitot does not sandbox their filesystem, environment or network access. + +Your Controller defines what approval, correctness or completion means. Pitot +validates and transports its answer; it does not establish those meanings. + +## Reference + +- [Host setup and all supported adapters](docs/hosts.md): Codex, Qwen, DeepSeek, + Claude, Cursor, Devin, Copilot, Gemini, Kimi, OpenCode and Pi. +- [Runtime and configuration](docs/runtime.md): tenant fragments, readiness, + manual runtime, explicit requests, fallback, privacy and lifecycle limits. +- [Wire contract](docs/protocol.md): schemas, framing and tested SDK constraints. +- [Python SDK](sdk/python/README.md), [TypeScript SDK](sdk/typescript/README.md) + and [Rust SDK](sdk/rust/README.md). +- [Contributing](CONTRIBUTING.md): development checks and adapter admission. + +Apache-2.0. See [LICENSE](LICENSE). diff --git a/docs/deepseek-adapter.md b/docs/deepseek-adapter.md index f8cb7b7..650640e 100644 --- a/docs/deepseek-adapter.md +++ b/docs/deepseek-adapter.md @@ -5,21 +5,32 @@ pre-execute middleware. The supported release is `@deepseek-ai/dsh@0.2.0-rc.2`, a developer preview. It requires Node.js 22.19 or newer in the 22 series, or Node.js 24 or newer. -Install that release and initialize the integration in your project: +Install that release, then create and register a sample shell Controller in your +project before wiring the host: ```sh npm install --global @deepseek-ai/dsh@0.2.0-rc.2 +pitot init --template shell-policy --language go --dir shell-policy pitot init --host deepseek pitot doctor --host deepseek ``` -Start a prompt with your Pitot configuration: +The template requires Go and writes its Controller registration under +`.pitot/conf.d/`. Its demonstration policy denies commands containing +`PITOT_DENY_ME`; replace it with your application policy. Host initialization +alone does not register a Controller. See [runtime configuration](runtime.md) +for the tenant fragment format. + +With provider credentials configured, start a prompt: ```sh pitot dev --host deepseek --prompt "Inspect this project" -- dsh ``` -Configure a shell Controller in `pitot.json`. Shell execution requires an +This command can contact the configured model provider. For real CLI processes +without real model calls, use the [local supervisory-control replay](../examples/supervisory-control/README.md). + +Shell execution requires an explicit, matching Controller reply. A missing runtime, unavailable Controller, timeout, malformed reply, or mismatched identity blocks execution, including when the runtime's ordinary fallback policy permits an action. An explicit diff --git a/docs/hosts.md b/docs/hosts.md new file mode 100644 index 0000000..fc1223e --- /dev/null +++ b/docs/hosts.md @@ -0,0 +1,376 @@ +# Host setup and boundaries + +## Supported hosts + +Every host below normalizes its native blocking boundary to a `shell` action and +passes Pitot's language-neutral decoder conformance suite. The E2E column marks +adapters exercised by the [cross-platform checks](https://github.com/operatorstack/pitot/actions/workflows/pitot-e2e.yml); +the Platforms column names the finite test targets, not a universal platform or +host-version guarantee. Kimi +additionally has an in-repo, no-model test that asserts the full allow **and** +deny control path end to end. + +| Host | Blocking boundary | Hook wiring | Platforms | Verified in this repo | +| ------------------ | ------------------------------------ | ------------------------------------------------ | ------------------------------------------------- | --------------------------------------- | +| Codex | `PreToolUse` | bridge (`integrations/codex`) | Ubuntu · macOS · Windows | decoder + E2E | +| Qwen Code | `PreToolUse` | bridge (`integrations/qwen`) | Ubuntu · macOS · Windows | decoder + E2E | +| DeepSeek Harness | `tools/pre-execute` / `bash`, `pwsh` | native middleware ([setup](deepseek-adapter.md)) | Ubuntu · macOS · Windows | decoder + E2E | +| Claude | `PreToolUse` | native settings hook | Ubuntu · macOS · Windows | decoder + E2E | +| Cursor | `beforeShellExecution` | bridge (`integrations/cursor`) | Ubuntu · macOS · native Windows with PowerShell 7 | decoder + E2E | +| Devin | `session/request_permission` (ACP) | stateful ACP transport | Ubuntu · macOS · Windows | decoder + E2E | +| GitHub Copilot CLI | `PreToolUse` | bridge (`integrations/copilot`) | Ubuntu · macOS · Windows | decoder + E2E | +| Gemini | `BeforeTool` | bridge (`integrations/gemini`) | Ubuntu · macOS · Windows | decoder + E2E | +| Kimi Code | `PreToolUse` / Bash | native `config.toml` | Ubuntu · macOS · Windows | decoder + E2E + allow/deny control test | +| OpenCode | `PreToolUse` | bridge (`integrations/opencode`) | Ubuntu · macOS · Windows | decoder + E2E | +| Pi | `tool_call` | extension (`integrations/pi`) | Ubuntu · macOS · Windows | decoder + E2E | + +"Decoder" means Pitot correctly normalizes that host's payload into the stable +event envelope. It does not claim Pitot judges whether any command is safe — that +decision belongs to your Controller. Hosts run natively on Windows. Cursor's +Windows hook bridge requires PowerShell 7. + +## Shared setup and diagnostics + +The per-host hooks below wire each agent's native blocking boundary to Pitot. +Hosts whose hook config lives **in the repository** are wired by Pitot itself: + +```bash +pitot init --host claude # also: cursor, codex, gemini +``` + +That writes exactly one marker-owned entry into the host's repo config +(`.claude/settings.json`, `.cursor/hooks.json`, `.codex/hooks.json`, +`.gemini/settings.json`), pointing at the repo shim — foreign entries are +never touched, and the committed fragment `.pitot/hooks/.fragment.json` +witnesses what was installed. `pitot doctor --host HOST` reports the entry as +FOUND, MISSING, or DRIFTED against that witness; `pitot doctor --host HOST +--fix` restores a drifted entry (Pitot's own entries only — that is the single +mutation doctor ever performs, and only on request). + +For Codex, Qwen and DeepSeek, doctor also checks the executable and the +integration's static setup on macOS, Linux and Windows. It reports independent +problems together. A `static setup: PASS` is scoped to the inspected files and +dependencies; `live enforcement: UNVERIFIED` means no agent action or Controller +decision was exercised. Codex session trust and normalized hook hashes still +require inspection in `/hooks`; stored trust that doctor cannot compare returns +a nonzero result. Unsupported command forms, conflicting Qwen hook definitions +and additional Codex configuration layers also require native inspection. + +With `PITOT_RUNTIME` unset, command hooks remain observe-only. DeepSeek native +middleware instead requires a live Controller reply. With it set, doctor checks +the selected runtime's identity through its local health endpoint. It does not +follow redirects, use an HTTP proxy, contact remote endpoints or submit a test +action. A healthy runtime does not establish that a Controller is available. +Plain doctor does not modify settings or grant trust; `--fix` performs the +supported wiring step, then runs the diagnostic checks separately. + +Hosts configured at **user level** (Kimi, Copilot, Qwen) or via **plugin +files** (OpenCode, Pi) keep the one-time manual edit below — Pitot does not +edit files outside the repository; `pitot init --host kimi` prints the exact +snippet. Once wired, both `pitot dev` and the manual runtime flow use the same +hook. + +## Codex and Claude + +```bash +pitot init --host claude +pitot init --host codex +``` + +Both init commands wire a `PreToolUse` hook (matcher `Bash`) whose command runs the repo +shim: `"$CLAUDE_PROJECT_DIR"/.pitot/bin/pitot hook claude` (Codex uses the +repo-relative equivalent plus a PowerShell `commandWindows`). Exit `0` allows; +exit `2` blocks with the Controller's reason. + +For Codex, the project-local `.codex/` configuration must be trusted. Open +`/hooks` in the Codex CLI to review and trust the exact hook definition. New or +changed non-managed hooks are skipped until trusted; writing the hook file +alone does not activate supervision. See [Codex hook trust](https://learn.chatgpt.com/docs/hooks#review-and-trust-hooks). + +## Qwen Code + +On macOS or Linux, copy `integrations/qwen/PreToolUse` to a stable absolute +path and make it executable (`chmod +x /path/to/PreToolUse`). This bridge needs +Bash, `python3`, and `pitot` on the Qwen process PATH; an explicit Pitot executable +can also be passed as its single argument. Add this command hook to +`~/.qwen/settings.json`: + +```json +{ + "hooks": { + "PreToolUse": [ + { + "matcher": "^run_shell_command$", + "hooks": [{ "type": "command", "command": "/path/to/PreToolUse" }] + } + ] + } +} +``` + +On Windows, copy `integrations/qwen/PreToolUse.cjs` to a stable path. Use Node +explicitly, with the bridge path and the real `pitot.exe` path as arguments. +For example, replace the placeholders below with your installed paths in +`%USERPROFILE%\.qwen\settings.json`: + +```json +{ + "hooks": { + "PreToolUse": [ + { + "matcher": "^run_shell_command$", + "hooks": [ + { + "type": "command", + "command": "node.exe \"C:/Tools/Pitot/PreToolUse.cjs\" \"C:/Tools/Pitot/pitot.exe\"" + } + ] + } + ] + } +} +``` + +The Qwen process must resolve `node.exe` on PATH. This example uses ordinary +native Windows shell handling; Git Bash/MSYS environments need separate native +inspection. Run `pitot doctor --host qwen` after configuring either bridge, and +review any conflicting or disabled hooks before relying on the setup. + +The matcher uses Qwen's runtime shell tool ID. Compatibility aliases such as +`Bash` match exactly; anchoring that alias as `^Bash$` does not select the shell +tool. + +Qwen sends the native JSON payload on standard input. The bridge returns its +native structured allow or deny decision and preserves the Controller reason. +See the official +[Qwen Code hooks guide](https://qwenlm.github.io/qwen-code-docs/en/users/features/hooks/). + +## DeepSeek Harness + +Use the official supported Harness release and the native middleware described +in [DeepSeek setup](deepseek-adapter.md). Register a shell Controller under +`.pitot/conf.d/` separately from host wiring. For example, from the project root: + +```sh +pitot init --template shell-policy --language go --dir shell-policy +pitot init --host deepseek +pitot doctor --host deepseek +``` + +Start a single prompt with `pitot dev --host deepseek --prompt "YOUR PROMPT" -- dsh`. +Ordinary prompts use the configured provider; the [local replay example](../examples/supervisory-control/README.md) +is the no-real-model-call demonstration. DeepSeek's native `bash` and `pwsh` +tools require an explicit matching Controller reply. Other tools are not +mediated by this integration. Existing Harness sandbox constraints still apply. + +## Kimi Code + +Install Kimi Code on macOS or Linux using its official installer: + +```bash +curl -fsSL https://code.kimi.com/kimi-code/install.sh | bash +kimi --version +``` + +On Windows, use the official PowerShell installer: + +```powershell +irm https://code.kimi.com/kimi-code/install.ps1 | iex +kimi --version +``` + +Connect Kimi Code's blocking shell boundary to Pitot in +`~/.kimi-code/config.toml`: + +```toml +[[hooks]] +event = "PreToolUse" +matcher = "Bash" +command = "pitot hook kimi" +``` + +Kimi sends the hook payload to Pitot on standard input. Pitot exits `0` when +the request is accepted and `2` when input is malformed or the configured +Controller denies the action. A normal Kimi prompt can contact its configured model provider. The following +is usage syntax, not an offline enforcement test: + +```bash +kimi -p "Show the repository status" +``` + +See the [Kimi Code documentation](https://www.kimi.com/code/docs/en/) for CLI +authentication, configuration, and hook behavior. + +## GitHub Copilot CLI + +Copy `integrations/copilot/PreToolUse` to a stable executable path (or use +`PreToolUse.ps1` on Windows), then add the following Claude-compatible hook to +`~/.copilot/settings.json`: + +```json +{ + "hooks": { + "PreToolUse": [ + { + "matcher": "Bash", + "hooks": [{ "type": "command", "command": "/path/to/PreToolUse" }] + } + ] + } +} +``` + +The bridge keeps the blocking payload on Pitot's standard `hook_event_name` +and `tool_input.command` boundary and returns Copilot's structured native deny +reason when a Controller rejects the command. See the official +[Copilot CLI hooks reference](https://docs.github.com/en/copilot/reference/hooks-reference). + +## Cursor + +```bash +pitot init --host cursor +``` + +Writes the committed bridge `.pitot/bin/hooks/cursor-beforeShellExecution` +(which execs the repo shim and returns Cursor's native `permission: "deny"` +envelope, including the Controller message) and points `.cursor/hooks.json` at +it with `failClosed: true`. See Cursor's +[hooks documentation](https://cursor.com/docs/agent/hooks). + +The generated Cursor bridge above uses a Unix shell. For native Windows, use +`integrations/cursor/beforeShellExecution.ps1` with PowerShell 7 and keep +`failClosed: true` in the hook configuration. Put the checked PowerShell 7 +directory on Cursor's process `PATH` and invoke that same `pwsh.exe` explicitly +in the hook command. Cursor uses PowerShell to read and pass the hook payload; +Windows PowerShell 5.1 can corrupt Unicode before the bridge receives it and is +not supported by this bridge. The verification harness configures both shell +invocations without changing the user's `PATH` or installing WSL. + +## Gemini + +```bash +pitot init --host gemini +``` + +Writes the committed bridge `.pitot/bin/hooks/gemini-BeforeTool` and registers +it as a `BeforeTool` command hook for `run_shell_command`. The bridge +translates Pitot rejection into Gemini's structured `decision: "deny"` and +`reason` response so the model receives the blocked tool result. See the +[Gemini CLI hooks reference](https://geminicli.com/docs/hooks/reference/). + +## Pi + +Copy `integrations/pi/pitot.ts` into `~/.pi/agent/extensions/pitot.ts` (or the +repository-local `.pi/extensions/` directory). The extension converts Pi's +blocking `tool_call` event into Pitot's stable envelope and returns Pi's native +`block` response when Pitot rejects the request. See the official +[Pi extensions documentation](https://pi.dev/docs/latest/extensions). + +## Devin + +Devin needs no hook file. Pitot speaks to it over the Agent Client Protocol, +launching `devin acp` as a stdio JSON-RPC server and correlating each +`tool_call` command by its tool-call ID: + +```bash +pitot dev --host devin -- devin -p "Run: echo hello" +``` + +To attach to an already-running runtime, use the explicit single-prompt +surface: + +```bash +pitot acp devin --runtime "$PITOT_RUNTIME" --prompt "Run: echo hello" +``` + +Pitot maps an allow decision to ACP's one-shot `allow_once` and a deny to +`reject_once`; it never selects a persistent option such as `allow_always` or a +bypass mode. Attestation comes from the Controller receipts and canary rather +than a lifecycle-hook witness. This initial adapter is single-prompt; resume and +multi-turn Devin sessions are not yet supported. See +[docs/devin-adapter-research.md](devin-adapter-research.md) for why ACP, +not hooks, is the supervised boundary. + +## OpenCode + +The supplied [OpenCode plugin](../integrations/opencode/server.ts) forwards the +native shell `execute.before` boundary to `pitot hook opencode`. Configure the +plugin in OpenCode's settings; Pitot does not install or edit user-level plugin +settings. The checked harness shows the adapter wiring in +[real_agent_driver.py](../tests/real_agent_driver.py). A plugin error becomes the +native rejected tool result. This is shell supervision, not coverage of every tool. + +## Threat model + +Pitot mediates the host's wired boundary and nothing else. Understanding where +that boundary ends is part of using it correctly. + +- **Only the wired boundary is mediated.** Pitot sees an action when the host + actually routes it through the configured hook or ACP transport. An action + the host takes through a path you did not wire is not observed. `pitot doctor +--host HOST` reports whether the boundary is present, and for repo-owned + hosts whether the entry has drifted. +- **User-level hook configs are user-editable.** For hosts wired at user level + (Kimi, Copilot, Qwen) the hook lives in a file the user owns and can change + or remove. Pitot does not police edits outside the repository; it reports the + current state through `pitot doctor`. +- **Without a runtime, hooks only observe.** A `pitot hook` invocation with no + `PITOT_RUNTIME` (or `--runtime`) selected records the action and exits + allowing it — observation-only, for backwards compatibility. It now prints a + one-line notice so this mode is never silent, and `pitot doctor` flags a host + that is wired but has no runtime. Once a runtime is explicitly selected, + transport or authentication failure blocks the controllable action rather + than falling open. +- **DeepSeek requires a live Controller reply.** Its native shell middleware + blocks execution without a selected runtime or an explicit matching reply, + even if ordinary runtime fallback defaults allow. This differs from the + observation-only behavior of command hooks with no runtime selected. +- **Hosts retain their own bypass options.** Some hosts expose persistent or + bypass permissions (for example Devin's ACP `allow_always` and switch-bypass + modes). Pitot never selects them — it uses only the one-shot `allow_once` / + `reject_once` options — but a human operating the host directly still can. +- **Kimi executes the action if its hook process crashes or times out.** This + is Kimi's host semantics, not a Pitot decision, and Pitot cannot override it: + a supervisory-control analysis of the Kimi lifecycle shows a reachable + transition from the pending state straight to an unsupervised execution when + the hook fails open, which no supervisor placed at the boundary can prevent. + `pitot doctor --host kimi` states this plainly. If your policy cannot + tolerate fail-open execution, prefer a fail-closed host (such as Cursor, + wired with `failClosed: true`) or Devin's ACP transport. +- **Codex hook failures can leave an action unsupervised.** A functioning + synchronous hook can block an action with a Controller's explicit denial or + Pitot's rejection of an unavailable selected runtime. Codex can continue + the tool when the hook cannot start, crashes, times out, or returns invalid + output. That host behavior is separate from a Controller's decision; a + passing allow/deny test does not guarantee that hook execution failures block + tools. Untrusted or changed hooks can also be skipped until reviewed. + +- **Qwen hook failures can leave an action unsupervised.** The pinned Qwen + release treats some hook startup errors, timeouts, and invalid output as + nonblocking. A functioning bridge can return a denial, but that does not + guarantee an action is blocked when the bridge itself fails. Valid structured + denial output and the host's blocking exit status are separate from these + nonblocking failure paths. + +Within an admitted runtime request, Pitot correlates a terminal resolution. +Host hook startup, trust, bypass and transport behavior remain separate boundaries. +A successful allow/deny check does not prove every possible host action is supervised. + +## Host admission criteria + +A host earns a supervised adapter only when its blocking boundary can complete +Pitot's causal loop: a proposed command must reach a Controller, an allow or +deny decision must apply **before** the command runs, and — critically — a +denied action must return to the model so the agent can continue from the +blocked outcome rather than halting. This last requirement, deny-continuation, +is what distinguishes a supervisable boundary from one that can only abort. + +Devin is the worked example. Its lifecycle hooks (`PreToolUse`, +`PermissionRequest`) apply a denial but end the turn instead of handing the +rejected outcome back to the model, so they do not complete the loop in +non-interactive mode. Its Agent Client Protocol surface does: the client +selects `reject_once`, the canary never executes, and Devin makes a follow-up +model request carrying the rejection and continues. Pitot therefore ships Devin +over ACP and does not ship the hook wiring. The full investigation, including +the content-safe evidence receipt, is in +[docs/devin-adapter-research.md](devin-adapter-research.md). diff --git a/docs/protocol.md b/docs/protocol.md new file mode 100644 index 0000000..7a7f846 --- /dev/null +++ b/docs/protocol.md @@ -0,0 +1,126 @@ +# Wire contract + +The canonical contract is the versioned schema plus the raw transport rules and +request state machine. Tested runner agreement is scoped to the conformance +corpus and supported targets; generated types alone do not validate every input. + +## Language-neutral by design + +Pitot's reference implementation is Go. Its public contract is not. + +Compatibility is defined by: + +- versioned JSON Schemas; +- newline-delimited JSON framing; +- explicit request/response state machines; +- capability and projection declarations; and +- language-neutral conformance fixtures. + +If a program can read JSON Lines from standard input, it can be a Consumer. A Controller must also follow the raw framing, registration and correlation +rules below; a schema-valid JSON value alone is not sufficient. + +Official client libraries are optional conveniences—not a prerequisite and +not the source of protocol truth. + +## Event envelope + +Events identify their schema, host, source, and observation quality. +`session_id` is optional, opaque metadata supplied by the host; Pitot preserves +it without generating a substitute when it is unavailable. Group sessions by +`host.name` and `session_id`, rather than treating IDs from different hosts as +one namespace. The generated per-action ID correlates control replies and is +separate from the host's session identity. Session metadata is retained by every +content projection and does not authenticate a Controller. + +This example shows an envelope with session metadata: + +```json +{ + "pitot_version": "1", + "id": "evt_01J...", + "type": "action.requested", + "time": "2026-07-19T16:05:00Z", + "host": { + "name": "cursor", + "adapter_version": "1.0.0" + }, + "session_id": "sess_42", + "action": { + "id": "act_7f2", + "kind": "shell" + }, + "content": { + "mode": "sha256", + "sha256": "9f2c00a18f8164a5f13127c1c38e48e67d97db8a03ccdc0c7a1cb86842f90d85" + }, + "observation": { + "source": "host_hook", + "fidelity": "direct" + } +} +``` + +Adapters preserve host capability differences. A normalized field is never +presented as directly observed when the host omitted it or Pitot inferred it. + +## Runtime envelope admission + +The reference runtime validates event semantics before any Consumer delivery. +Events require version `1`, a nonempty host name, and an observation source of +`host_hook` or `host_event` with fidelity `direct` or `estimated`. Action events +also require a nonempty action ID and kind. Optional event metadata and +Controller-owned request data retain their existing meaning. + +Content representations are mutually exclusive: + +- `full` carries one JSON string containing the measured bytes, with no digest; +- `sha256` carries a 64-digit hexadecimal digest, with no full content; +- `omit` carries neither representation. + +Contradictory or malformed content is rejected before fanout, including events +submitted directly through the Go Manager. Projection validates content again +before using an already-projected representation. Validation errors contain no +payload or envelope identifiers. Valid envelopes retain the existing projection, +Controller correlation, timeout and unavailable-default behavior. + +The shared typed-envelope vectors are in +`conformance/fixtures/envelopes.jsonl`. This admission gate does not enable new +event delivery capabilities: runtime ingress still accepts only +`action.requested`. SDK readers apply the wire contract below before invoking handlers. + +## SDK wire contract + +The canonical v1 value constraints are published in [schema/v1.json](../schema/v1.json). +Generated SDK types are structural conveniences; the Go, Python, TypeScript and +Rust runners validate raw input before constructing handler values. Optional +metadata may be absent, but explicit null is rejected for typed string/object +fields. Controller-owned `data` may contain any JSON value, including null. + +The JSON-Lines transport profile adds constraints that JSON Schema cannot check +after decoding: + +- UTF-8 and Unicode scalar strings, with unique decoded member names in every + object, including nested Controller data; +- one object per LF-delimited record, exact-case envelope fields, and no trailing + JSON or unknown envelope/typed-object fields; +- at most 4 MiB per record, excluding LF (CR counts), and 128 nested containers; +- token counts spelled as JSON integers, from 0 through 9007199254740991. Decimal + and exponent spellings are rejected even when their mathematical value is + integral; this preserves typed-reader compatibility and avoids silent rounding. + +Blank JSON whitespace lines are ignored. CRLF and a final nonempty record without +LF are accepted. Malformed and oversized lines are skipped with content-safe +diagnostics; oversized records are drained so the next valid record can proceed. +Controller results are validated before stdout, and absent response messages are +omitted. FIFO handler ordering is preserved, including asynchronous TypeScript. +Python admission does not depend on assertions and also runs under `python -O`. + +The shared raw corpus is [conformance/fixtures/wire.jsonl](../conformance/fixtures/wire.jsonl). +Tests distinguish decoded JSON Schema constraints from transport constraints and +drive the real runners. Arbitrary Controller data still uses each language's +native JSON representation: numeric precision/range and absent/null distinctions +are not made identical by the SDKs. Generated constructors used directly do not +apply the runner admission gate. The Go API retains platform-sized `int` counters; +verification and shipped targets are 64-bit, while 32-bit callers have a narrower +native integer range. The runtime continues to deliver only +`action.requested` events; accepting `model.usage` in SDKs adds no host capability. diff --git a/docs/runtime.md b/docs/runtime.md new file mode 100644 index 0000000..24507b9 --- /dev/null +++ b/docs/runtime.md @@ -0,0 +1,406 @@ +# Runtime, configuration and privacy + +## Install + +A released CLI can pin its release in the repository. `pitot init` creates the +following files alongside your tenant fragments, leaving existing files intact: + +- `.pitot/version` — one committed semver line, the only version authority +- `.pitot/bin/pitot` (+ `pitot.ps1`) — a committed shim that reads the pin, + hydrates exactly that release into a per-user cache + (SHA-256 verified against the published `checksums.txt` during download), + and executes it + +Source builds without an injected release version, including ordinary +`go install` builds, still write the shims but skip a missing `.pitot/version` +and print a notice. Use a released CLI to initialize the pin, or explicitly pin +a published release before using the shims. A source build can run directly +without a repository pin. + +On macOS and Linux, the default cache slot is +`~/.cache/pitot//-/`. `XDG_CACHE_HOME` changes the cache base; +`PITOT_CACHE_DIR` overrides the Pitot cache directory. On Windows, the default +is `%LOCALAPPDATA%\pitot\\windows-\`; `PITOT_CACHE_DIR` overrides +that directory too. + +Fresh clones, CI, and cloud agents can run `.pitot/bin/pitot` (or the PowerShell +shim on Windows). It hydrates when the selected cache slot is absent and reuses +an existing slot. There is no +fallback to whatever binary happens to be on PATH; a missing release with no +network fails closed with a named error, and `PITOT_NO_HYDRATE=1` makes +hydration cache-only. + +**Upgrades are a reviewed diff.** `pitot upgrade` verifies the new release, +re-checks every tenant fragment against it, and rewrites the one pin line — +nothing else. Commit that diff and every clone hydrates the new version on +its next invocation; roll back by reverting it. + +```bash +pitot upgrade --check # report pinned vs latest +pitot upgrade # hydrate, validate tenants, rewrite .pitot/version +``` + +For a global CLI convenience (running `pitot init` in new repos), grab a +release binary or build from source: + +```bash +go install github.com/operatorstack/pitot/cmd/pitot@latest +``` + +Typed SDKs install from our own registry through the distribution front door +— never public npm/PyPI, pinned to the CLI's version: + +```bash +pitot install typescript # scoped .npmrc + @operatorstack/pitot@ +pitot install python # .pitot/registry + operatorstack-pitot== +``` + +(`install typescript` creates a minimal private `package.json` when the +directory has none — npm honors a project registry config only inside a +project.) + +Inspect the local boundary, pin, and cache state: + +```bash +pitot doctor +``` + +## Start a dev runtime + +First register a Controller and wire the host as described in [host setup](hosts.md). +From the project root: + +```sh +pitot init --template shell-policy --language go --dir shell-policy +``` + +Templates are `shell-policy`, `release-approval`, `blank-controller` and +`blank-consumer`. The first four runnable languages are Python, TypeScript, Go +and Rust. The generated project must have its dependencies available before +its configured command can run. The sample shell policy is only a canary tripwire. + +`pitot dev --host HOST -- COMMAND ARGS` discovers fragments, starts their +Consumers and Controllers, waits for readiness and launches the agent with +`PITOT_RUNTIME` set. It prints Controller decisions. DeepSeek also requires +`--prompt`; Devin uses ACP. See [host setup](hosts.md) for each launch form. +Readiness establishes a runtime endpoint, not live hook enforcement or trust. + +The runtime directory is private to each invocation and removed on exit. +On Windows, standard npm Node launchers for Codex and Qwen are recognized without +batch evaluation. Custom `.cmd` wrappers are rejected with explicit Node +entrypoint guidance. Prefer `-- COMMAND ARGS` for paths and arguments containing +spaces; `--exec` splits its value on whitespace. + +The generic subprocess launcher uses a two-second captured-output drain limit. +A descendant retaining an output pipe can produce an incomplete-output error +rather than successful completion. DeepSeek and Devin use separate ACP lifecycle +handling; see their host documentation for cancellation limits. Cleanup releases +the dev runtime but does not promise to kill arbitrary descendant processes. + +## Multiple tools, one Pitot + +Configuration is tenant-partitioned: every tool or person that registers +processes with Pitot owns exactly one fragment in `.pitot/conf.d/`, and no +tenant ever edits another tenant's file. The effective configuration is the +deterministic merge of all fragments in filename order: + +``` +.pitot/ + conf.d/ + boatstack.yaml # a tool's controller, written by its installer + interlock.yaml # another tool's controller, different request kind + my-policy.yaml # your own, scaffolded by `pitot init` +``` + +Each fragment is a complete, strictly parsed mini-config declaring +`controllers:` and/or `consumers:`. Merge rules: + +- **Consumers always compose.** Any number of tenants can observe + `action.requested` events. +- **A request kind has one owner.** Two fragments claiming the same kind (for + example `shell`) fail discovery with an error naming both files — a loud, + attributable conflict instead of two tools silently fighting over one + blocking hook. Controller and consumer ids must also be unique across + fragments. +- **`requires_protocol: "1"`** optionally pins the protocol version a fragment + was written against; a fragment this binary cannot honor fails discovery. +- **`dir:`** sets a process's working directory (relative to the repository + root), so each tenant's command stays project-relative: + +```yaml +controllers: + shell: + id: local-shell-policy + command: ["go", "run", "main.go"] + dir: "shell-policy" + deadline_ms: 2000 + on_timeout: deny + on_unavailable: deny +``` + +Installing a second Pitot-based tool is therefore additive by construction: it +drops its own fragment next to yours, `pitot run`/`pitot dev` merge them, and +uninstalling it is deleting its fragment. + +## Advanced: manual runtime + +`pitot dev` is the recommended path. If you need to manage the runtime yourself +(for example, sharing one runtime across several long-lived agent sessions), +start it with repository-owned configuration and an owner-only runtime +descriptor: + +```bash +export PITOT_RUNTIME="${XDG_RUNTIME_DIR:-${TMPDIR:-/tmp}}/pitot/project.json" +pitot run --runtime "$PITOT_RUNTIME" +``` + +With no `--config`, `pitot run` discovers and merges the repository's +`.pitot/conf.d/` fragments. Pass `--config PATH` to override discovery with one +explicit file (useful for tests and ad-hoc runtimes). + +One runtime owns each descriptor path. Pitot holds an operating-system lock on +the owner-only `PATH.lock` companion through startup, serving and cleanup. Leave +that empty file in place; the lock is released automatically on shutdown or a +process crash. Competing starters fail without changing the owner's descriptor. +An uncertain health result (including a timeout) also refuses takeover. A stale +descriptor is recovered only after a direct loopback connection refusal. +Older Pitot versions do not share this lock, so stop them before starting a new +runtime at the same path. + +Start coding-agent CLIs from the same environment. Their `pitot hook HOST` +commands discover the authenticated runtime through `PITOT_RUNTIME`. Without +that variable or `--runtime PATH`, command hooks remain observation-only for backwards +compatibility. Once a runtime is explicitly selected, transport or +authentication failure causes Pitot to reject the controllable action; whether +the host honors a failed hook remains a host boundary. DeepSeek native middleware +requires a live explicit Controller response. + +On Windows, set the descriptor in the launching PowerShell session: + +```powershell +$env:PITOT_RUNTIME = Join-Path $env:LOCALAPPDATA "Pitot\project.json" +pitot run --runtime $env:PITOT_RUNTIME +``` + +## Two small programs + +### 1. Audit shell requests without recording commands + +Save the Python program below as `examples/action-audit.py` in your project, +then register this Consumer in `.pitot/conf.d/action-audit.yaml` with an `omit` +content projection: + +```yaml +consumers: + - id: action-audit + command: ["python3", "./examples/action-audit.py"] + events: ["action.requested"] + projection: + content: omit +``` + +Pitot writes newline-delimited JSON to the program's standard input: + +```json +{ + "pitot_version": "1", + "type": "action.requested", + "host": { "name": "claude" }, + "action": { "id": "act_7f2", "kind": "shell" }, + "content": { "mode": "omit" }, + "observation": { "source": "host_hook", "fidelity": "direct" } +} +``` + +The Consumer is ordinary Python—no Pitot SDK required: + +```python +import json +import sys + +for line in sys.stdin: + event = json.loads(line) + print(json.dumps({ + "host": event["host"]["name"], + "action_id": event["action"]["id"], + "kind": event["action"]["kind"], + }), file=sys.stderr) +``` + +The command is projected out before bytes enter the Consumer pipe. Consumer +failure cannot allow or deny the waiting host action. + +### 2. Let a skill request approval + +An application or skill can make a synchronous request to an already-running +runtime. Register its Controller before sending the request: + +```bash +pitot request release.approval --data '{"release":"v1.4.0"}' --runtime "$PITOT_RUNTIME" +``` + +Register one Controller for that request kind: + +```yaml +controllers: + release.approval: + id: local-approval + command: ["go", "run", "./examples/local-approval"] + deadline_ms: 2000 + on_timeout: deny + on_unavailable: deny +``` + +The Controller receives: + +```json +{ + "pitot_version": "1", + "type": "control.requested", + "kind": "release.approval", + "action_id": "act_7f2", + "data": { "release": "v1.4.0" } +} +``` + +It checks its own source of truth and returns one correlated response: + +```json +{ + "pitot_version": "1", + "type": "control.response", + "controller_id": "local-approval", + "action_id": "act_7f2", + "outcome": "allow", + "message": "v1.4.0 is approved for publication." +} +``` + +Pitot validates the controller identity, action ID, deadline, schema, and +single-response rule before carrying the answer back. Pitot does not know what +“approved” means; the Controller owns that definition. + +The reference local-approval program handles one request and exits; it is a +wire illustration, not a persistent approval service. Its example rule allows +`release.approval` without an external approval check. Use the SDK runner and +your own source of truth for an application Controller. + +## Trusted local processes + +Pitot uses supervised local processes in v1. It starts declared Consumers and +Controllers itself, applies each projection before bytes enter the child pipe, +and exposes only a loopback endpoint authenticated by the owner-only runtime +descriptor. Pitot does not pass the capability token directly to child processes or include it +in its diagnostics. These are trusted programs running as the launching user: +they may read the owner-only descriptor and access its authenticated endpoint. +Pitot does not sandbox them or restrict their filesystem, environment, or network +access. Descriptor permissions exclude other users, not programs running as the owner. + +## Controller guarantees + +For Go applications, `runtime.Start` validates the protocol and all process +registrations before launching any child. Empty embedded runtimes remain valid; +file configuration still requires at least one registration. An explicit request +can use any configured nonempty kind, and its unavailable/timeout defaults may +allow or deny. `DeliverControlledEvent` separately requires a live Controller +reply for an action event. + +Synchronous hooks are control channels: the host is blocked waiting for an +answer. Pitot makes that privilege explicit. + +- Exactly one Controller may register for a request kind. +- Registration is static, auditable configuration. +- The registration declares a deadline and unavailable/timeout defaults. +- The Controller deadline covers waiting for a busy Controller, writing its + request, and waiting for the response. A caller deadline can shorten that + budget; canceled requests do not wait indefinitely behind another request. +- If a write is interrupted, Pitot stops that Controller because its input may + contain a partial frame. Later requests use `on_unavailable` until the runtime + is restarted. A timeout after a complete write leaves the Controller running; + late replies cannot resolve a different action. +- Every response is bound to its pending action and Controller identity. +- Late, stale, duplicate, mismatched, and malformed responses are rejected. +- An admitted pending runtime request receives one terminal resolution. +- Registration and its configuration fingerprint appear in diagnostics. + +The bridge mechanically enforces the declared default. It contains no approval, +safety, completion, or shipping policy engine. + +## Boundary faults + +Pitot distinguishes a broken measurement boundary from a judgment about the +work: + +```json +{ + "pitot_version": "1", + "type": "boundary.fault", + "host": "cursor", + "action_id": "act_7f2", + "reason": "empty-command" +} +``` + +Reason codes never contain prompts, commands, tool inputs, or outputs. A malformed +hook payload produces this diagnostic on operator stderr, and the hook returns a +blocked outcome before runtime delivery. The registered Controller does not +receive this fault as a request. Applications that need to react to diagnostics +must arrange a separate input path; host handling of the failed hook still +determines whether execution is prevented. + +## Privacy model + +Pitot is local and storage-free by default. + +- Raw host payloads terminate inside the adapter. +- Content projection is `full`, `sha256`, or `omit` per Consumer. +- Projection happens before delivery, not inside downstream applications. +- No network exporter is enabled implicitly. +- Pitot does not retain an event history unless a configured Consumer does. +- Consumer output is not wired to the Controller response channel. +- Pitot-generated event delivery and response rejection diagnostics omit envelope + identifiers and content. Configured process labels may appear in diagnostics. + Controller stderr and both Consumer output streams are forwarded to operator + stderr; this application-controlled output may contain sensitive information. + +Projection controls command content in each delivered event. It preserves other +metadata and does not prevent a trusted program from obtaining content elsewhere. +SHA-256 supports equality checks and comparison against guessed content; it is +not encryption. An existing digest can be narrowed to `omit`; requests that need +unavailable full content or a missing digest are rejected. Absent content stays +absent under every projection policy. + +Consumer delivery is bounded and best effort. Queue saturation and unavailable +processes produce diagnostics. An unexpected process exit or pipe-write failure +reports once that observation delivery may be incomplete, even without another +event. Queue acceptance and a successful pipe write are not acknowledgements that +an application processed the event. Pitot does not retry or retain dropped +observations; a Consumer can implement its own retention. Expected runtime +shutdown is quiet. Consumer pipe writes run separately from Controller resolution; +a slow operator diagnostic sink can still block existing delivery diagnostics. + +## Applications + +Use action events for reports and diagnostics, and registered requests for human +approval, delivery verification or state-dependent policy. The +[supervisory-control example](../examples/supervisory-control/README.md) shows a +shared program checking current tested artifact revisions. + +`model.usage` exists in the schema and SDKs, but the runtime does not deliver it. +The [token-meter wire example](../examples/token-meter/main.go) needs records +supplied by your application. Token accounting and cross-agent memory require +additional data sources; Pitot does not supply them automatically. + +## What Pitot does not decide + +Pitot does not define whether: + +- work is correct or complete; +- an action is safe; +- a person granted approval; +- evidence satisfies a requirement; +- a claim is valid; or +- something may be shipped. + +Those meanings belong to your Controller. **Pitot reports. Your controller +decides.** diff --git a/examples/doc.go b/examples/doc.go index 4cbb8d5..36289ac 100644 --- a/examples/doc.go +++ b/examples/doc.go @@ -1,10 +1,12 @@ -// Package examples documents the two reference programs that demonstrate the -// Consumer/Controller distinction, mirroring the public README: +// Package examples includes two small JSON-Lines reference programs: // -// token-meter a passive Consumer that counts tokens without recording prompts +// token-meter a passive Consumer for application-supplied model.usage events // local-approval a Controller that returns one correlated response per request // -// Each is a standalone main package under this directory so it can be built and -// run without a Pitot SDK. A Consumer reads JSON Lines from standard input; a -// Controller returns one schema-valid response on its output channel. +// The shipped runtime delivers action.requested events, not model token usage. +// Connecting token-meter to it therefore does not collect token counts. Each +// example is a standalone main package and reads one JSON record per line from +// standard input. Controller responses must use the wire framing and match the +// pending action and registered Controller; schema validity alone is insufficient. +// The supervisory-control directory contains a larger offline delivery example. package examples diff --git a/examples/token-meter/main.go b/examples/token-meter/main.go index 7751538..7f33350 100644 --- a/examples/token-meter/main.go +++ b/examples/token-meter/main.go @@ -1,6 +1,7 @@ -// Command token-meter is a passive Consumer: it sums token usage from the Pitot -// event stream without ever recording prompts. It needs no Pitot SDK — it only -// reads newline-delimited JSON from standard input. +// Command token-meter sums application-supplied model.usage events without +// recording prompts. The shipped Pitot runtime delivers action.requested events, +// not model usage, so registering this Consumer does not collect token counts. +// This example reads one JSON record per line from standard input. package main import ( diff --git a/internal/testrole/main.go b/internal/testrole/main.go index 3ae07cb..6f7c5b8 100644 --- a/internal/testrole/main.go +++ b/internal/testrole/main.go @@ -19,8 +19,15 @@ func main() { receipt := flag.String("receipt", "", "append-only receipt path") nonce := flag.String("nonce", "", "E2E nonce") mode := flag.String("mode", "route", "route, timeout, malformed, mismatch, allow, or deny") + requireTerminal := flag.Bool("require-terminal", false, "require terminal output for the canary") flag.Parse() if *role == "canary" { + if *requireTerminal { + if err := checkTerminal(os.Stdout); err != nil { + fmt.Fprintln(os.Stderr, err) + os.Exit(2) + } + } if err := canary(*receipt, flag.Args()); err != nil { fmt.Fprintln(os.Stderr, err) os.Exit(2) @@ -38,6 +45,14 @@ func main() { controller(*id, *receipt, *nonce, *mode) } +func checkTerminal(output *os.File) error { + info, err := output.Stat() + if err != nil || info.Mode()&os.ModeCharDevice == 0 { + return fmt.Errorf("testrole: canary requires terminal output") + } + return nil +} + func canary(receipt string, args []string) error { if receipt == "" || len(args) != 2 { return fmt.Errorf("testrole: canary requires --receipt, marker, and nonce") diff --git a/internal/testrole/main_test.go b/internal/testrole/main_test.go index 4d28e78..0d3c25b 100644 --- a/internal/testrole/main_test.go +++ b/internal/testrole/main_test.go @@ -2,10 +2,33 @@ package main import ( "os" + "os/exec" "path/filepath" + "strings" "testing" ) +func TestCanaryTerminalGuardRejectsPipeBeforeReceipt(t *testing.T) { + if os.Getenv("PITOT_TEST_CANARY_CHILD") == "1" { + os.Args = []string{"testrole", "--role", "canary", "--receipt", os.Getenv("PITOT_TEST_RECEIPT"), "--require-terminal", "PITOT_ALLOW", "nonce"} + main() + return + } + path := filepath.Join(t.TempDir(), "receipt") + command := exec.Command(os.Args[0], "-test.run=^TestCanaryTerminalGuardRejectsPipeBeforeReceipt$") + command.Env = append(os.Environ(), "PITOT_TEST_CANARY_CHILD=1", "PITOT_TEST_RECEIPT="+path) + output, err := command.CombinedOutput() + if err == nil || !strings.Contains(string(output), "canary requires terminal output") { + t.Fatalf("expected terminal rejection, got %s, %v", output, err) + } + if strings.Contains(string(output), "PITOT_CANARY_RESULT") { + t.Fatal("rejected canary emitted a result") + } + if _, err := os.Stat(path); !os.IsNotExist(err) { + t.Fatalf("rejected canary created a receipt: %v", err) + } +} + func TestCanaryWritesNonceBoundReceipt(t *testing.T) { path := filepath.Join(t.TempDir(), "canary.jsonl") if err := canary(path, []string{"PITOT_ALLOW", "nonce"}); err != nil { diff --git a/scripts/pitot_adapter_supervisor.py b/scripts/pitot_adapter_supervisor.py index 064fe86..a55dd0e 100644 --- a/scripts/pitot_adapter_supervisor.py +++ b/scripts/pitot_adapter_supervisor.py @@ -464,15 +464,13 @@ def readme_block(manifest: dict[str, object]) -> str: return "\n".join( [ README_START, - '

', - ' Pitot agent E2E', - "

", + "[Agent checks on main](https://github.com/operatorstack/pitot/actions/workflows/pitot-e2e.yml?query=branch%3Amain)", "", - '

Every supervised adapter must pass a binary-observed prompt → real action boundary → projected Consumer → Controller allow/deny → tool-result loop on Ubuntu, macOS, and Windows.

', + "Every supervised adapter must pass a binary-observed prompt → real action boundary → projected Consumer → Controller allow/deny → tool-result loop on Ubuntu, macOS, and Windows.", "", - f'

Supervised adapters: {labels}

', + f"Supervised adapters: {labels}", "", - '

Supervised runtime capabilities: action control · Consumer delivery · explicit request

', + "Supervised runtime capabilities: action control · Consumer delivery · explicit request", README_END, ] ) diff --git a/sdk/csharp/README.md b/sdk/csharp/README.md index 387b44b..fe7580c 100644 --- a/sdk/csharp/README.md +++ b/sdk/csharp/README.md @@ -2,7 +2,7 @@ **Status: types-only preview. Not a shipping SDK.** `Pitot/Types.cs` holds the generated C# data types for the Pitot wire protocol, emitted by -[`scripts/generate_types.sh`](../../../scripts/generate_types.sh) from the same +[`scripts/generate_types.sh`](../../scripts/generate_types.sh) from the same schema as every other binding. They let you _read the shapes_ — they do **not** yet give you a runnable Controller or Consumer. @@ -11,8 +11,8 @@ What is **not** here yet, and what a first-class SDK requires: - no build manifest (`.csproj` / solution), so the package does not build; - no `runner` (the stdio JSON-Lines Controller/Consumer loop the other SDKs ship); - no entry in the cross-language conformance suite - ([`../tests/test_runners.py`](../tests/test_runners.py)), which is what proves a - binding's wire behavior is identical to the others; + ([`../tests/test_runners.py`](../tests/test_runners.py)), which compares + bindings against a shared set of wire-contract cases; - not covered by the version-lockstep gate — it carries no release version. The first-class, runnable SDKs are **Python**, **TypeScript**, **Go**, and diff --git a/sdk/java/README.md b/sdk/java/README.md index 5ab58db..28aae16 100644 --- a/sdk/java/README.md +++ b/sdk/java/README.md @@ -2,7 +2,7 @@ **Status: types-only preview. Not a shipping SDK.** These files are the generated Java data types for the Pitot wire protocol (`com.operatorstack.pitot.*`), emitted -by [`scripts/generate_types.sh`](../../../scripts/generate_types.sh) from the same +by [`scripts/generate_types.sh`](../../scripts/generate_types.sh) from the same schema as every other binding. They let you _read the shapes_ — they do **not** yet give you a runnable Controller or Consumer. @@ -11,8 +11,8 @@ What is **not** here yet, and what a first-class SDK requires: - no build manifest (`pom.xml` / `build.gradle`), so the package does not build; - no `runner` (the stdio JSON-Lines Controller/Consumer loop the other SDKs ship); - no entry in the cross-language conformance suite - ([`../tests/test_runners.py`](../tests/test_runners.py)), which is what proves a - binding's wire behavior is identical to the others; + ([`../tests/test_runners.py`](../tests/test_runners.py)), which compares + bindings against a shared set of wire-contract cases; - not covered by the version-lockstep gate — it carries no release version. The first-class, runnable SDKs are **Python**, **TypeScript**, **Go**, and diff --git a/tests/endpoint-provenance.json b/tests/endpoint-provenance.json index 38ec0e5..7787de9 100644 --- a/tests/endpoint-provenance.json +++ b/tests/endpoint-provenance.json @@ -1726,531 +1726,531 @@ "artifacts": [ { "agent": "claude", - "archive_sha256": "f63af455fb9c074795414f342d7e69bf64c4a1381d670a041aca11a32ad20799", - "artifact_id": 11828311257, + "archive_sha256": "36be4cf3841e751f16e7231275810246f9fdc3abb7bcbcea6f098534f3ab6ed2", + "artifact_id": 11833245141, "capture_sha256": "5708547a3442f7789838ecf0a8840e6d71e9f38f0e0406ea8f0f6f22b1898a1a", - "check_run_id": 115382969402, + "check_run_id": 115423685577, "conclusion": "success", - "job_id": 115382969402, + "job_id": 115423685577, "platform": "macos", "runner": "macos-latest", - "runner_id": 1000036656, + "runner_id": 1000036859, "runner_labels": [ "macos-latest" ], - "runner_name": "GitHub Actions 1000036656" + "runner_name": "GitHub Actions 1000036859" }, { "agent": "claude", - "archive_sha256": "3fd9ff9a7dc6b30ac4be525b9f9cdbde4def3cd2439d5b135e9c86e479ca0bc3", - "artifact_id": 11827166318, + "archive_sha256": "a44581460939e002625d88c0554bb07325185562d17862db28df270be904cd43", + "artifact_id": 11832535820, "capture_sha256": "adcd14006488d4df783429269c2c1a4cfc107045503a5980da0dca6a9422a787", - "check_run_id": 115382970219, + "check_run_id": 115423684546, "conclusion": "success", - "job_id": 115382970219, + "job_id": 115423684546, "platform": "ubuntu", "runner": "ubuntu-latest", - "runner_id": 1000036637, + "runner_id": 1000036840, "runner_labels": [ "ubuntu-latest" ], - "runner_name": "GitHub Actions 1000036637" + "runner_name": "GitHub Actions 1000036840" }, { "agent": "claude", - "archive_sha256": "fc341eb19bfa1da285d1a149eefaa26b2af2fc6705beabb39cd698b812c434c4", - "artifact_id": 11827351192, + "archive_sha256": "03086898b6e667af89bb3d2dd8da3cc267fb1326429e8c46ec44e83cd0dec9fd", + "artifact_id": 11832446455, "capture_sha256": "53f68de22f5190e01a339b2960dbb38b2bd649a4004b0e4e5f28cd3d08bf9925", - "check_run_id": 115382969535, + "check_run_id": 115423685190, "conclusion": "success", - "job_id": 115382969535, + "job_id": 115423685190, "platform": "windows", "runner": "windows-latest", - "runner_id": 1000036641, + "runner_id": 1000036850, "runner_labels": [ "windows-latest" ], - "runner_name": "GitHub Actions 1000036641" + "runner_name": "GitHub Actions 1000036850" }, { "agent": "codex", - "archive_sha256": "b8294f203cc67612e1d25771857810d2333dc77b100d94d3f6453cd629ad01ca", - "artifact_id": 11827737672, + "archive_sha256": "5135d2a62aa1fcb07026966fe3d3014fb83bab965020e3c2dfd1a88d3560c817", + "artifact_id": 11833120361, "capture_sha256": "d55f571d3db096ba3333ce57423f3af12804d62b82df2a3e5606f9f69c4b01b8", - "check_run_id": 115382969854, + "check_run_id": 115423685044, "conclusion": "success", - "job_id": 115382969854, + "job_id": 115423685044, "platform": "macos", "runner": "macos-latest", - "runner_id": 1000036665, + "runner_id": 1000036857, "runner_labels": [ "macos-latest" ], - "runner_name": "GitHub Actions 1000036665" + "runner_name": "GitHub Actions 1000036857" }, { "agent": "codex", - "archive_sha256": "db521c6e957a2812fdcb8df14ed5b76e92826682ae0f32014017147cf12b3536", - "artifact_id": 11825989885, + "archive_sha256": "c98f0ece39dea609798f39871bbcbfc97b55874095415fadd8fb9b8f7648a1fa", + "artifact_id": 11832321819, "capture_sha256": "60977d4a8c9cb5d3f2ee339756837d0262587072583ed1ca7a6148728889289d", - "check_run_id": 115382969645, + "check_run_id": 115423684963, "conclusion": "success", - "job_id": 115382969645, + "job_id": 115423684963, "platform": "ubuntu", "runner": "ubuntu-latest", - "runner_id": 1000036625, + "runner_id": 1000036847, "runner_labels": [ "ubuntu-latest" ], - "runner_name": "GitHub Actions 1000036625" + "runner_name": "GitHub Actions 1000036847" }, { "agent": "codex", - "archive_sha256": "2298b7b45e19a485d03d62265692de8867a68af6f671267a117d258383ebfbce", - "artifact_id": 11827026854, + "archive_sha256": "493560c622fb04a2a6fcfe94fd0ab34989423fdc6ede90ff56892e3e2073a065", + "artifact_id": 11832845115, "capture_sha256": "4944bae99385b50a7d5821b108398d10dbe07e6324168b22f5a353d42e50f57d", - "check_run_id": 115382969735, + "check_run_id": 115423684778, "conclusion": "success", - "job_id": 115382969735, + "job_id": 115423684778, "platform": "windows", "runner": "windows-latest", - "runner_id": 1000036631, + "runner_id": 1000036837, "runner_labels": [ "windows-latest" ], - "runner_name": "GitHub Actions 1000036631" + "runner_name": "GitHub Actions 1000036837" }, { "agent": "copilot", - "archive_sha256": "8f5cc2856d749b51db88d786999a4b179e43e3b510fd20f148135e399203da14", - "artifact_id": 11828166126, + "archive_sha256": "d7b60e331bf49b0d769394d11c76c2c937c481a3e6f9703abd728c9d6e0c9f44", + "artifact_id": 11832198429, "capture_sha256": "d03816631d68c5cc22e18102e6d06bc01e3a435f583648ef7b3e871bcb31b847", - "check_run_id": 115382969673, + "check_run_id": 115423684461, "conclusion": "success", - "job_id": 115382969673, + "job_id": 115423684461, "platform": "macos", "runner": "macos-latest", - "runner_id": 1000036654, + "runner_id": 1000036836, "runner_labels": [ "macos-latest" ], - "runner_name": "GitHub Actions 1000036654" + "runner_name": "GitHub Actions 1000036836" }, { "agent": "copilot", - "archive_sha256": "90bd46cf7243d7c021980236d9e066dba035b5571dc5133deac49e3f63af9fb8", - "artifact_id": 11827425815, + "archive_sha256": "71e5f04f932f53f89d6b3b6b2576b1872a7a818b11f7eec82f71d0aaedba57bc", + "artifact_id": 11832013564, "capture_sha256": "587960da7ac17da700faeae5d9fa3a4dff9ce729fdaffb01d4cd52b9fdd0f9ef", - "check_run_id": 115382969662, + "check_run_id": 115423684931, "conclusion": "success", - "job_id": 115382969662, + "job_id": 115423684931, "platform": "ubuntu", "runner": "ubuntu-latest", - "runner_id": 1000036630, + "runner_id": 1000036841, "runner_labels": [ "ubuntu-latest" ], - "runner_name": "GitHub Actions 1000036630" + "runner_name": "GitHub Actions 1000036841" }, { "agent": "copilot", - "archive_sha256": "4085338ffb055cb6e36992a7d648aeca9022df64923156e22122ad790bdba1d3", - "artifact_id": 11826814537, + "archive_sha256": "72028babd923e8e24090e2c699417cabfedc626ed2ce5ffc9a986b32dd0f5b13", + "artifact_id": 11831489171, "capture_sha256": "e7a4c28bad0899a9774f4bcdc264c8d453fd80afe6a03304a490d0bd457aa086", - "check_run_id": 115382970069, + "check_run_id": 115423684582, "conclusion": "success", - "job_id": 115382970069, + "job_id": 115423684582, "platform": "windows", "runner": "windows-latest", - "runner_id": 1000036638, + "runner_id": 1000036842, "runner_labels": [ "windows-latest" ], - "runner_name": "GitHub Actions 1000036638" + "runner_name": "GitHub Actions 1000036842" }, { "agent": "cursor", - "archive_sha256": "4825ed8c40f83aa7de88e667dac9457826bd4a8ab383ebd629b2631f823fb77f", - "artifact_id": 11827908685, + "archive_sha256": "52e56369d2385bbd82ff013da04a414c5b27d913772c2066fd11486bbc7c479c", + "artifact_id": 11832765489, "capture_sha256": "f6005b969def3bb81ec2aabcedae6f32e662aee91bb60b68a08bb72efb5fc49b", - "check_run_id": 115382970284, + "check_run_id": 115423684330, "conclusion": "success", - "job_id": 115382970284, + "job_id": 115423684330, "platform": "macos", "runner": "macos-latest", - "runner_id": 1000036670, + "runner_id": 1000036855, "runner_labels": [ "macos-latest" ], - "runner_name": "GitHub Actions 1000036670" + "runner_name": "GitHub Actions 1000036855" }, { "agent": "cursor", - "archive_sha256": "f0dd69ed724e94ce338a50d4947199047e2eab3dfccc2c08719fbb079f4988b1", - "artifact_id": 11827905113, + "archive_sha256": "a353836b0f0979c1cab60f0f64cc0639393e69cf73a268e66be302f08dac01d8", + "artifact_id": 11831868600, "capture_sha256": "286e0e249de8e190f8b45153b5c28eb5eedfcc3104b05b751370063dad5df502", - "check_run_id": 115382970492, + "check_run_id": 115423684291, "conclusion": "success", - "job_id": 115382970492, + "job_id": 115423684291, "platform": "ubuntu", "runner": "ubuntu-latest", - "runner_id": 1000036639, + "runner_id": 1000036831, "runner_labels": [ "ubuntu-latest" ], - "runner_name": "GitHub Actions 1000036639" + "runner_name": "GitHub Actions 1000036831" }, { "agent": "cursor", - "archive_sha256": "708d3f69247c162e9ed640f71ef8bfac2795e42885836415a20ea8914b1aac5f", - "artifact_id": 11826549724, + "archive_sha256": "6e2b48b43dad5d2706166c660c4d12d781900ee71e3f68a80316ad8101629916", + "artifact_id": 11832560768, "capture_sha256": "efeaa573c8a48652a6a8a4ee3e664075565712cf131cd1fc58a30e46f2552027", - "check_run_id": 115382969983, + "check_run_id": 115423684436, "conclusion": "success", - "job_id": 115382969983, + "job_id": 115423684436, "platform": "windows", "runner": "windows-latest", - "runner_id": 1000036635, + "runner_id": 1000036828, "runner_labels": [ "windows-latest" ], - "runner_name": "GitHub Actions 1000036635" + "runner_name": "GitHub Actions 1000036828" }, { "agent": "deepseek", - "archive_sha256": "0194594f793cd1ec71d048f589846db30dda4341ba99f60bcfe33a68267cc213", - "artifact_id": 11828701320, + "archive_sha256": "654eeb4a8ef2e72318bc90415b5b10d3b10936cf48392ebbad95218c51080a99", + "artifact_id": 11831854997, "capture_sha256": "6ad19c7fd4ddbe24c4981206cf8cd3e444e17fa51069c22f49637c682da09c22", - "check_run_id": 115382969125, + "check_run_id": 115423685509, "conclusion": "success", - "job_id": 115382969125, + "job_id": 115423685509, "platform": "macos", "runner": "macos-latest", - "runner_id": 1000036663, + "runner_id": 1000036858, "runner_labels": [ "macos-latest" ], - "runner_name": "GitHub Actions 1000036663" + "runner_name": "GitHub Actions 1000036858" }, { "agent": "deepseek", - "archive_sha256": "2548fa43e4d60e56c6bba3283989a506b7c22e94638dbc3098def100388babda", - "artifact_id": 11827011718, + "archive_sha256": "85af67c828a809553015a3ff1464f5772d6d66e326873f957881f5b58ba356f2", + "artifact_id": 11832835302, "capture_sha256": "6dbb15f123bae299a14b3bfdfb8f63a2b9ef149815c88db98dfec9cec7a6b916", - "check_run_id": 115382969303, + "check_run_id": 115423685310, "conclusion": "success", - "job_id": 115382969303, + "job_id": 115423685310, "platform": "ubuntu", "runner": "ubuntu-latest", - "runner_id": 1000036623, + "runner_id": 1000036852, "runner_labels": [ "ubuntu-latest" ], - "runner_name": "GitHub Actions 1000036623" + "runner_name": "GitHub Actions 1000036852" }, { "agent": "deepseek", - "archive_sha256": "014bbca7fd294fa035e4e601f060bad22f6a76a17071674b9ac5d1ed9f233ed0", - "artifact_id": 11828421446, + "archive_sha256": "7c499db7fe04a4125c09667b8a65b4abc666bd622db74e1d0adbe67b2c022d08", + "artifact_id": 11833550307, "capture_sha256": "605b85be3d155c233da5c1b3f46d043a3c14b487a50e11322ddf276195182abb", - "check_run_id": 115382969593, + "check_run_id": 115423685228, "conclusion": "success", - "job_id": 115382969593, + "job_id": 115423685228, "platform": "windows", "runner": "windows-latest", - "runner_id": 1000036632, + "runner_id": 1000036843, "runner_labels": [ "windows-latest" ], - "runner_name": "GitHub Actions 1000036632" + "runner_name": "GitHub Actions 1000036843" }, { "agent": "devin", - "archive_sha256": "8b0b872e02ac8175f8a3c4a50adfd9ca12d4d1422937c54e090479bb0eaa49e2", - "artifact_id": 11827384216, + "archive_sha256": "7bdc83c50db96717f3bf101cb2ebc080b196a0351918313b05ca012ef4148e41", + "artifact_id": 11832233266, "capture_sha256": "20cb48009cffb16f0c37a6c73fdba0ee4bf1a038e38d8db21265eb2802f3d238", - "check_run_id": 115382969418, + "check_run_id": 115423684316, "conclusion": "success", - "job_id": 115382969418, + "job_id": 115423684316, "platform": "macos", "runner": "macos-latest", - "runner_id": 1000036669, + "runner_id": 1000036833, "runner_labels": [ "macos-latest" ], - "runner_name": "GitHub Actions 1000036669" + "runner_name": "GitHub Actions 1000036833" }, { "agent": "devin", - "archive_sha256": "99ae51f2a249283897ce88f35b2e5393c8d855bd9d6e13469aa4c2ad4b096bda", - "artifact_id": 11826754522, + "archive_sha256": "274d87e65c4ac1cea88d0a16f261053cc7701660ec135992abda74395d8d2df1", + "artifact_id": 11831698697, "capture_sha256": "4436b85b9122e967a08ac2d956a41dfe0244f581ec3f27c285ee91f33cf7b0c6", - "check_run_id": 115382970184, + "check_run_id": 115423684315, "conclusion": "success", - "job_id": 115382970184, + "job_id": 115423684315, "platform": "ubuntu", "runner": "ubuntu-latest", - "runner_id": 1000036636, + "runner_id": 1000036832, "runner_labels": [ "ubuntu-latest" ], - "runner_name": "GitHub Actions 1000036636" + "runner_name": "GitHub Actions 1000036832" }, { "agent": "devin", - "archive_sha256": "cf49b3e9d06b0b20e30d5a380535c092694e463c28c7792fa2e363960376bab3", - "artifact_id": 11827425857, + "archive_sha256": "5dca994e370d325aaf4013e3d0f296c2d9e5e4c7886e32a61cfa1f7d2d9fc698", + "artifact_id": 11832845264, "capture_sha256": "03837ff7295608b1f14dd6011ca3fc49a0d7204f9bc83748b6374b5e184d9537", - "check_run_id": 115382969498, + "check_run_id": 115423685130, "conclusion": "success", - "job_id": 115382969498, + "job_id": 115423685130, "platform": "windows", "runner": "windows-latest", - "runner_id": 1000036627, + "runner_id": 1000036849, "runner_labels": [ "windows-latest" ], - "runner_name": "GitHub Actions 1000036627" + "runner_name": "GitHub Actions 1000036849" }, { "agent": "gemini", - "archive_sha256": "b3ef0f9626cbf06050a3848c35d4d8e1861617e3f16aa71e1bd750b2ebbd05ce", - "artifact_id": 11828472131, + "archive_sha256": "398f491e006e72b31587ea55e4bc6841ba6052e04c522d99e2bba32ead0a0d38", + "artifact_id": 11832193451, "capture_sha256": "1d30d5dbb4ab3a6f2c23605492e4d7912960eaf7d1225748b61eba669625588e", - "check_run_id": 115382970652, + "check_run_id": 115423684738, "conclusion": "success", - "job_id": 115382970652, + "job_id": 115423684738, "platform": "macos", "runner": "macos-latest", - "runner_id": 1000036666, + "runner_id": 1000036835, "runner_labels": [ "macos-latest" ], - "runner_name": "GitHub Actions 1000036666" + "runner_name": "GitHub Actions 1000036835" }, { "agent": "gemini", - "archive_sha256": "f71a630a81e2c5a616743106255c6c41ec94a0b3495dff5a8152aa69b0b3134a", - "artifact_id": 11826829288, + "archive_sha256": "b0e6ae17cbb2fd3762b20b4c2b10df3aad808bed5564c886b6ae38d9137a0282", + "artifact_id": 11832640616, "capture_sha256": "ba385c092e030a6efddab5d3a07abce7d84c23995b282e3c691d4f2d14868e06", - "check_run_id": 115382969265, + "check_run_id": 115423685531, "conclusion": "success", - "job_id": 115382969265, + "job_id": 115423685531, "platform": "ubuntu", "runner": "ubuntu-latest", - "runner_id": 1000036619, + "runner_id": 1000036853, "runner_labels": [ "ubuntu-latest" ], - "runner_name": "GitHub Actions 1000036619" + "runner_name": "GitHub Actions 1000036853" }, { "agent": "gemini", - "archive_sha256": "033da9b4993f2b0ec723187ca144214f7a143c02bd1f2c3f3ab251f1febb4134", - "artifact_id": 11827006832, + "archive_sha256": "211e917b3993e8f307b53ac465101f7bde8ce4fbb6ff4c7c71896b783e508805", + "artifact_id": 11831749098, "capture_sha256": "fae8bc7c58bb6d0bc7b63e6861104efa96e82375726259134ec7a46e11f39e0d", - "check_run_id": 115382969324, + "check_run_id": 115423684995, "conclusion": "success", - "job_id": 115382969324, + "job_id": 115423684995, "platform": "windows", "runner": "windows-latest", - "runner_id": 1000036621, + "runner_id": 1000036838, "runner_labels": [ "windows-latest" ], - "runner_name": "GitHub Actions 1000036621" + "runner_name": "GitHub Actions 1000036838" }, { "agent": "kimi", - "archive_sha256": "2bb720b436c88500c38d47cae62c3d0f55cd91d44416d86912b7d565f5417077", - "artifact_id": 11828246653, + "archive_sha256": "4ab36c7331ee4fff0bd2ea05cc78ce0f86ff8896c5607f32ef802743567bd9c0", + "artifact_id": 11832208773, "capture_sha256": "7e8e7a839318423e8817ac5238c794de45da9dc6391c3aa034e4d073900ac50d", - "check_run_id": 115382968798, + "check_run_id": 115423684348, "conclusion": "success", - "job_id": 115382968798, + "job_id": 115423684348, "platform": "macos", "runner": "macos-latest", - "runner_id": 1000036662, + "runner_id": 1000036854, "runner_labels": [ "macos-latest" ], - "runner_name": "GitHub Actions 1000036662" + "runner_name": "GitHub Actions 1000036854" }, { "agent": "kimi", - "archive_sha256": "0851614410bd5cc345efe83cbf5f3e60b3df4cbb0f62330a664d0f90778f51d0", - "artifact_id": 11826913870, + "archive_sha256": "09ebcd78c6775c8effa5f888249862bf7a74ee1fcc6572bdfc595d5aab8aa6fc", + "artifact_id": 11832471262, "capture_sha256": "635636fc781e5e3834179e102386496cb15c1f995a297077504e13a57a14f73e", - "check_run_id": 115382970113, + "check_run_id": 115423685223, "conclusion": "success", - "job_id": 115382970113, + "job_id": 115423685223, "platform": "ubuntu", "runner": "ubuntu-latest", - "runner_id": 1000036628, + "runner_id": 1000036846, "runner_labels": [ "ubuntu-latest" ], - "runner_name": "GitHub Actions 1000036628" + "runner_name": "GitHub Actions 1000036846" }, { "agent": "kimi", - "archive_sha256": "720308ac5fe24c26c9e49b3a327af9fe0025e53caffb814505514f445ee572f1", - "artifact_id": 11826534967, + "archive_sha256": "1a24c5775ea321079389222c216fed332d3272b0f863acf64a9ccac4ee38ef38", + "artifact_id": 11831804004, "capture_sha256": "bbf937c664af66ff6badeb1650593fe60ecb7ed92d0601837f124fd95d596616", - "check_run_id": 115382969898, + "check_run_id": 115423685385, "conclusion": "success", - "job_id": 115382969898, + "job_id": 115423685385, "platform": "windows", "runner": "windows-latest", - "runner_id": 1000036634, + "runner_id": 1000036851, "runner_labels": [ "windows-latest" ], - "runner_name": "GitHub Actions 1000036634" + "runner_name": "GitHub Actions 1000036851" }, { "agent": "opencode", - "archive_sha256": "a5abd8bf1dbe279d748132acf77124d8b0d19eb3dcb08347a9337011f3b0662a", - "artifact_id": 11827004559, + "archive_sha256": "0bfeaf0969d3c011c15418a5524f175c505e25144ad1d1b084e709e832cf32ee", + "artifact_id": 11832376977, "capture_sha256": "40c1b102d236d40cfb9f4002c58289e23540caed58ebda405bacf22f316dcf76", - "check_run_id": 115382969827, + "check_run_id": 115423684837, "conclusion": "success", - "job_id": 115382969827, + "job_id": 115423684837, "platform": "macos", "runner": "macos-latest", - "runner_id": 1000036667, + "runner_id": 1000036856, "runner_labels": [ "macos-latest" ], - "runner_name": "GitHub Actions 1000036667" + "runner_name": "GitHub Actions 1000036856" }, { "agent": "opencode", - "archive_sha256": "198e012d3f3f61de7fa07750544a82cc2600d19e2cd2e490e777a93c33ef2524", - "artifact_id": 11827361085, + "archive_sha256": "4297a5a9512a738b6d0fb66e89318f154021f3f36b6be298d9264d032920788d", + "artifact_id": 11831444198, "capture_sha256": "97ae1278b1c7aaeb080cf4a1ba29e7be3329bcef9d74a69edb8cb401f67d1ac2", - "check_run_id": 115382970552, + "check_run_id": 115423684672, "conclusion": "success", - "job_id": 115382970552, + "job_id": 115423684672, "platform": "ubuntu", "runner": "ubuntu-latest", - "runner_id": 1000036640, + "runner_id": 1000036839, "runner_labels": [ "ubuntu-latest" ], - "runner_name": "GitHub Actions 1000036640" + "runner_name": "GitHub Actions 1000036839" }, { "agent": "opencode", - "archive_sha256": "222aa4bdda9af1dd42b5ff9c332f6a06a04db47d9b9153139901327d683b7e82", - "artifact_id": 11826899155, + "archive_sha256": "6a2fcf78b5785133c73334007a2aac8161298823e87fd68fbd46cebdebd5cfd1", + "artifact_id": 11831569102, "capture_sha256": "525d48117f8840f28ecfc871960dad8b4cd8b4302d3027a9f9a1dfb69a338c1c", - "check_run_id": 115382969716, + "check_run_id": 115423684965, "conclusion": "success", - "job_id": 115382969716, + "job_id": 115423684965, "platform": "windows", "runner": "windows-latest", - "runner_id": 1000036633, + "runner_id": 1000036848, "runner_labels": [ "windows-latest" ], - "runner_name": "GitHub Actions 1000036633" + "runner_name": "GitHub Actions 1000036848" }, { "agent": "pi", - "archive_sha256": "07ab577dfbeb8d1ece4ea0e7eece9535331b62c9c7f0c2c218d1b7316bf56884", - "artifact_id": 11827008464, + "archive_sha256": "6a5f81b1ccbb7f04e6fd63f3e7a9e52e22e4829a596114738e903ed8ea565fdf", + "artifact_id": 11831544104, "capture_sha256": "3d50939aa99a10227ba51830a73526ecee0c1b3a3b17e5d1b96279b2d8f5cfc4", - "check_run_id": 115382969908, + "check_run_id": 115423684482, "conclusion": "success", - "job_id": 115382969908, + "job_id": 115423684482, "platform": "macos", "runner": "macos-latest", - "runner_id": 1000036657, + "runner_id": 1000036834, "runner_labels": [ "macos-latest" ], - "runner_name": "GitHub Actions 1000036657" + "runner_name": "GitHub Actions 1000036834" }, { "agent": "pi", - "archive_sha256": "56f9acb959c1ddd7549f695c781d86536fcadb8cd932b22826fe870056e5a2c6", - "artifact_id": 11826444492, + "archive_sha256": "5accd5f253b0d9fbf0b447ae18252afbceb2f1140cabb4972df5548a09fcef69", + "artifact_id": 11832268192, "capture_sha256": "0208904ba375b4f1402e643bb7bc1610933f8773a51c9b499f275c4b26ef9933", - "check_run_id": 115382969514, + "check_run_id": 115423685111, "conclusion": "success", - "job_id": 115382969514, + "job_id": 115423685111, "platform": "ubuntu", "runner": "ubuntu-latest", - "runner_id": 1000036620, + "runner_id": 1000036845, "runner_labels": [ "ubuntu-latest" ], - "runner_name": "GitHub Actions 1000036620" + "runner_name": "GitHub Actions 1000036845" }, { "agent": "pi", - "archive_sha256": "50f4ee7deb0c905861535d3dd8b2d289e8d5976ce18830de4a4d5fe105d148b5", - "artifact_id": 11826749694, + "archive_sha256": "878f34aa89e8354730b5fd9b132c53e762a2872545b7fa9350895aaa703d74f0", + "artifact_id": 11831703957, "capture_sha256": "e3764c13ab1be4a05d45e0c79b133e6009a3c0141cefd8b757ced474c74e5cd4", - "check_run_id": 115382970159, + "check_run_id": 115423684312, "conclusion": "success", - "job_id": 115382970159, + "job_id": 115423684312, "platform": "windows", "runner": "windows-latest", - "runner_id": 1000036629, + "runner_id": 1000036827, "runner_labels": [ "windows-latest" ], - "runner_name": "GitHub Actions 1000036629" + "runner_name": "GitHub Actions 1000036827" }, { "agent": "qwen", - "archive_sha256": "ccfb0fec7f0f4e9536bec0267353adf3151bce12f4d88df2b1365a9c009e7e3a", - "artifact_id": 11828076974, + "archive_sha256": "2f9ca885f89f0f1c4d0f511b8826e2f128aafc39d93d9030d33ebbbc200a0a27", + "artifact_id": 11833010791, "capture_sha256": "49dce0e5ac25dff70a306bcae73e396a7ca735cb8473594f53ac2d2b04ad743e", - "check_run_id": 115382970103, + "check_run_id": 115423684454, "conclusion": "success", - "job_id": 115382970103, + "job_id": 115423684454, "platform": "macos", "runner": "macos-latest", - "runner_id": 1000036660, + "runner_id": 1000036860, "runner_labels": [ "macos-latest" ], - "runner_name": "GitHub Actions 1000036660" + "runner_name": "GitHub Actions 1000036860" }, { "agent": "qwen", - "archive_sha256": "9fa4b95fd63831e6f5948f1c0377b1e31dbeef518ff0c55443e5f8c43b51d7e5", - "artifact_id": 11827450769, + "archive_sha256": "96e9ed23e52914dc8ad20b876783521514510d78622d0761c8b4b06e1474f6db", + "artifact_id": 11831139810, "capture_sha256": "5de080822b9a3934105b2b16d6ec3213e528a22f948fefbc07d8fb317b4797cc", - "check_run_id": 115382969317, + "check_run_id": 115423684340, "conclusion": "success", - "job_id": 115382969317, + "job_id": 115423684340, "platform": "ubuntu", "runner": "ubuntu-latest", - "runner_id": 1000036618, + "runner_id": 1000036830, "runner_labels": [ "ubuntu-latest" ], - "runner_name": "GitHub Actions 1000036618" + "runner_name": "GitHub Actions 1000036830" }, { "agent": "qwen", - "archive_sha256": "71b56cc9f20e1bdb9ade1ec4ee074b910ac399598d05e7ba2e74750014e9dae2", - "artifact_id": 11828160015, + "archive_sha256": "faf642c64cbb7e4844dfe46803e5455097f904160ccced752579a967f131253a", + "artifact_id": 11831244747, "capture_sha256": "0da935bc455440126660cfc9329fd955e92dea24dda67f290e8035875a6187ab", - "check_run_id": 115382969477, + "check_run_id": 115423684490, "conclusion": "success", - "job_id": 115382969477, + "job_id": 115423684490, "platform": "windows", "runner": "windows-latest", - "runner_id": 1000036622, + "runner_id": 1000036829, "runner_labels": [ "windows-latest" ], - "runner_name": "GitHub Actions 1000036622" + "runner_name": "GitHub Actions 1000036829" } ], "catalog_sha256": "f92926d7a643aed5dfb151ba8af1ff570f1a8e46a3587a91e2ae4498868abe82", @@ -2337,11 +2337,11 @@ }, { "path": "internal/testrole/main.go", - "sha256": "6d657eb85d8ddaff0ae5a3da281aa7aa0a4bdda860d179e99508110033765787" + "sha256": "70978b403c3bdb1332fe6eccf1cc5f460ab9675be9fbcea15030b3474453999d" }, { "path": "scripts/pitot_adapter_supervisor.py", - "sha256": "b0235e463d0d38a344c8aafa7be439f6aba1916c1b97db10126df43400f84b90" + "sha256": "9e33c65f01c2b174c3d51c73febe50775c5191bfd93d0f92985bd535abd1a218" }, { "path": "scripts/pitot_capture_source.py", @@ -2433,7 +2433,7 @@ }, { "path": "tests/real_agent_driver.py", - "sha256": "b22b0b5eb3aada118574941f8185aec5513c64972a9e58ee8ac2d9949b375bee" + "sha256": "01c8b5c4ecb626787059043b2a5641b229e3bec6a6131ac30d0a348ba30cadb6" }, { "path": "tests/run_e2e_report.py", @@ -2448,8 +2448,8 @@ "repository": "operatorstack/pitot", "repository_id": 1305805198, "run_attempt": 1, - "run_id": 38422394339, - "source_sha": "eb623f067914f978f38be71b908d388e6602c6af", + "run_id": 38434092561, + "source_sha": "926b633eea80f34e0f131ac5afa7a9f80841ce87", "workflow_id": 381486275, "workflow_path": ".github/workflows/pitot-e2e.yml" } diff --git a/tests/real_agent_driver.py b/tests/real_agent_driver.py index e54a8c8..aee3d5d 100644 --- a/tests/real_agent_driver.py +++ b/tests/real_agent_driver.py @@ -333,14 +333,18 @@ def windows_host() -> bool: return os.name == "nt" -def render_canary_command(executable: str, receipt: str, native_windows: bool) -> str: +def render_canary_command( + executable: str, receipt: str, native_windows: bool, require_terminal: bool = False +) -> str: if native_windows: executable = executable.replace("\\", "/") receipt = receipt.replace("\\", "/") - return subprocess.list2cmdline( - [executable, "--role", "canary", "--receipt", receipt] - ) - return shlex.join([executable, "--role", "canary", "--receipt", receipt]) + arguments = [executable, "--role", "canary", "--receipt", receipt] + if require_terminal: + arguments.append("--require-terminal") + return ( + subprocess.list2cmdline(arguments) if native_windows else shlex.join(arguments) + ) def cursor_powershell() -> str: @@ -700,6 +704,8 @@ def configure( f'"{witness_receipt}" "{nonce}" "{runtime_path}"' ) settings = hook_group("PreToolUse", "^run_shell_command$", hook_command) + # Use the host's terminal output drain; the canary rejects pipe fallback. + settings["tools"] = {"shell": {"enableInteractiveShell": True}} settings["modelProviders"] = { "openai": { "protocol": "openai", @@ -1291,6 +1297,33 @@ def validate_deepseek_native_results( } +def failed_canary_counts(path: Path, nonce: str) -> dict[str, object]: + unavailable = {"available": False} + try: + with path.open("rb") as stream: + raw = stream.read(65537) + except FileNotFoundError: + return {"available": True, "present": False, "total": 0, "allow": 0, "deny": 0} + except OSError: + return unavailable + if len(raw) > 65536: + return unavailable + try: + lines = raw.decode("utf-8").splitlines() + except UnicodeDecodeError: + return unavailable + allow, deny = f"PITOT_ALLOW {nonce}", f"PITOT_DENY {nonce}" + if len(lines) > 1000 or any(line not in (allow, deny) for line in lines): + return unavailable + return { + "available": True, + "present": True, + "total": len(lines), + "allow": lines.count(allow), + "deny": lines.count(deny), + } + + def validate_receipts( agent: dict[str, object], platform: str, @@ -1319,11 +1352,6 @@ def validate_receipts( witnesses = json_lines(witness_path) controller = json_lines(controller_path) consumers = json_lines(consumer_path) - canary = ( - canary_path.read_text(encoding="utf-8").splitlines() - if canary_path.is_file() - else [] - ) proxy_flags = ( "initial_prompt_observed", "allow_tool_call_response_emitted", @@ -1339,9 +1367,15 @@ def validate_receipts( or final_marker not in output or not all(proxy.get(flag) is True for flag in proxy_flags) ): + canary_counts = failed_canary_counts(canary_path, nonce) raise RuntimeError( - f"agent loop incomplete (exit={exit_code}, proxy={proxy}, witnesses={witnesses})\n{output[-4000:]}" + f"agent loop incomplete (exit={exit_code}, canary_counts={canary_counts}, proxy={proxy}, witnesses={witnesses})\n{output[-4000:]}" ) + canary = ( + canary_path.read_text(encoding="utf-8").splitlines() + if canary_path.is_file() + else [] + ) if proxy.get("nonce") != nonce: raise RuntimeError("proxy receipt escaped the nonce-bound real-agent session") if agent["id"] == "opencode": @@ -1756,6 +1790,7 @@ def run_session(args: argparse.Namespace) -> int: str(canary_executable), receipt_argument, windows_host() and not host_controls_wsl, + require_terminal=args.agent == "qwen", ) if ( args.agent in {"cursor", "deepseek"} diff --git a/tests/test_adapter_supervisor.py b/tests/test_adapter_supervisor.py index 4923a04..39e4cf6 100644 --- a/tests/test_adapter_supervisor.py +++ b/tests/test_adapter_supervisor.py @@ -527,6 +527,24 @@ def test_missing_script_fails(self): ) ) + def test_readme_status_is_plain_and_tracks_catalog(self): + manifest = _manifest() + manifest["agents"][0]["label"] = "Renamed adapter" + block = supervisor.readme_block(manifest) + self.assertNotIn( + "<", + block.replace(supervisor.README_START, "").replace( + supervisor.README_END, "" + ), + ) + self.assertNotIn("![", block) + self.assertIn("?query=branch%3Amain)", block) + labels = " · ".join(agent["label"] for agent in manifest["agents"]) + self.assertIn(f"Supervised adapters: {labels}", block) + self.assertIn("Ubuntu, macOS, and Windows", block) + self.assertTrue(block.startswith(supervisor.README_START)) + self.assertTrue(block.endswith(supervisor.README_END)) + def test_readme_and_workflow_drift_fail(self): with tempfile.TemporaryDirectory() as temp: root = Path(temp) diff --git a/tests/test_model_control_proxy.py b/tests/test_model_control_proxy.py index 37ed8ab..5277841 100644 --- a/tests/test_model_control_proxy.py +++ b/tests/test_model_control_proxy.py @@ -999,6 +999,15 @@ def test_windows_canary_command_is_shell_portable(self): self.assertIn("D:/a/temp/canary.jsonl", command) self.assertNotIn("\\", command) + def test_terminal_canary_flag_precedes_positional_markers(self): + for windows in (False, True): + command = driver.render_canary_command("canary", "receipt", windows, True) + self.assertTrue(command.endswith("--require-terminal")) + self.assertNotIn( + "--require-terminal", + driver.render_canary_command("canary", "receipt", windows), + ) + def test_anthropic_messages_emits_native_shell_tool(self): value = proxy.anthropic("Bash", "pitot-e2e-canary nonce", False) self.assertEqual(value["content"][0]["type"], "tool_use") diff --git a/tests/test_qwen_setup.py b/tests/test_qwen_setup.py index ddad0a5..6fd76bc 100644 --- a/tests/test_qwen_setup.py +++ b/tests/test_qwen_setup.py @@ -13,10 +13,10 @@ class QwenSetupTests(unittest.TestCase): def test_native_fixture_uses_documented_runtime_shell_matcher(self): section = ( - (driver.LAB / "README.md") + (driver.LAB / "docs/hosts.md") .read_text(encoding="utf-8") - .split("### Qwen Code\n", 1)[1] - .split("\n### ", 1)[0] + .split("## Qwen Code\n", 1)[1] + .split("\n## ", 1)[0] ) documented = json.loads(section.split("```json\n", 1)[1].split("```", 1)[0]) expected = documented["hooks"]["PreToolUse"][0]["matcher"] diff --git a/tests/test_receipt_validation.py b/tests/test_receipt_validation.py index af59ff0..c322731 100644 --- a/tests/test_receipt_validation.py +++ b/tests/test_receipt_validation.py @@ -216,6 +216,36 @@ def test_nonzero_exit_is_agent_loop_incomplete(self): "prompt", ) self.assertIn("agent loop incomplete", str(caught.exception)) + self.assertIn( + "canary_counts={'available': True, 'present': True, 'total': 1, 'allow': 1, 'deny': 0}", + str(caught.exception), + ) + + def test_failed_canary_observation_does_not_disclose_invalid_receipts(self): + for raw in ( + b"secret-sentinel", + b"\xff", + b"x" * 65537, + (f"PITOT_ALLOW {NONCE}\n" * 1001).encode(), + ): + with self.subTest(size=len(raw)): + self.fixture.canary.write_bytes(raw) + self.assertEqual( + driver.failed_canary_counts(self.fixture.canary, NONCE), + {"available": False}, + ) + + def test_failed_canary_observation_distinguishes_missing_and_empty(self): + self.fixture.canary.unlink(missing_ok=True) + self.assertEqual( + driver.failed_canary_counts(self.fixture.canary, NONCE), + {"available": True, "present": False, "total": 0, "allow": 0, "deny": 0}, + ) + self.fixture.canary.write_bytes(b"") + self.assertEqual( + driver.failed_canary_counts(self.fixture.canary, NONCE), + {"available": True, "present": True, "total": 0, "allow": 0, "deny": 0}, + ) class ValidateReceiptsACPBranches(unittest.TestCase):