[caballeto]

Summary

AI skills for synthetics (doc 42 §11, doc 37 "What the shipped skill must contain"). Stacked on #168 (base feat/synthetics-surfaces): review after #168, and retarget to main once #168 merges.

What's covered (doc 42 §11 map)

§11 item	Where
New devhelm-synthetics skill: preconditions (--version, doctor -o json), mode detection (YAML / Terraform / SDK / existing Playwright / checkly.config.ts), the loop validate --online → test → fix → deploy, exit-code table (4, 13, 14, 15, 16, 17, 18, 19), secrets rule, plan facts from account entitlements, the deploy-confirmation rule (exit 15 → show changes[], never add --yes for production), one entry file per check, @devhelm/synthetics waits for signup, investigate + classify, untrusted-input rule	skills/devhelm-synthetics/SKILL.md, references/writing-checks.md, references/investigating-runs.md, generated references/_generated/* (runs, revisions, bundles, script config, secret requests, runtimes)
Refresh configure/investigate/communicate/manage: remove commands and flags that don't exist, add synthetics verbs, the frozen D-2 exit codes and secrets values set	skills/devhelm-{configure,investigate,communicate,manage}/**
Skills generated from the manifest/spec	scripts/generate-skill-references.mjs (now also supports explicit schema lists and stamps metadata.version)
CI runs every fenced devhelm … example with --help and fails on an unknown command	scripts/check-skill-examples.mjs (npm run skills:check), run in npm test by test/skills/examples.test.ts. It also refuses unknown long flags
devhelm skills install installs devhelm-synthetics	SKILL_NAMES in src/lib/skills.ts (init uses the same list)
doctor warns when installed skill version ≠ CLI version	skillVersion() reads metadata.version; doctor skills check compares it with the CLI version (and flags edited copies)
Starter prompt in sync with init (doc 37 #9)	init's prompt now names devhelm-synthetics
devhelmhq/skill publishes the same set	scripts/sync-skill-repo.mjs <skill-checkout> (npm run skills:sync)

Version stamps: the generator stamps every SKILL.md with the package version; the npm version lifecycle script reruns it, so scripts/release.sh (which calls npm version) restamps the skills in the bump commit. A frontmatter test fails if a stamp lags.

Stale content fixed (examples): audit → audit events; status-pages incidents updates create → post-update; subscribers create/delete → add/remove; --published → --enabled; --environment → --env; incidents resolve --reason → --message; dependencies create → track; secrets values never as arguments; monitors list --status → --health. Removed (no command exists): monitors uptime, resource-groups uptime, incidents reopen, environments use, workspaces use (→ auth switch), plan-preview, webhooks deliveries, secrets get, domains get, init --from-platform.

Tests

• npm test: 61 files, 1236 tests, all pass locally (one run). The new examples.test.ts checks 209 fenced examples across 110 commands and runs --help for each command.
• npm run lint, npm run typecheck: clean.
• Real-API matrix: not applicable (docs + CLI-local checks only).

Self-review

One reviewer pass (Sonnet) across the CLI, skill and templates diffs: 0 P0. Fixed these P1s: the deploy --yes rule now treats every deploy as production unless the user says otherwise; billing multipliers are attributed to the right commands (test --repeat, trigger --region all, --max-runs); "Running" after deploy became a runs list check; the OTP example guards a missing code; the subjects are aligned; the bare-topic examples (devhelm audit) are now refused; sync-skill-repo refuses stale stamps or missing generated references; the doctor wording is clearer; the api-orders README covers API_URL.

Review follow-ups (P1/P2 not done):

• The checker covers fenced blocks only (as §11 says), not inline backtick mentions or \-continued lines.
• ExampleResult could become a discriminated union.
• The frontmatter stamp is written by the generator and read by skillVersion() with two small parsers; they could share one helper.
• Short flags (-o) aren't validated.
• --help runs are sequential (about 20 s).
• The templates repo has no CI.

Deviations from doc 42

• The examples check also refuses unknown long flags, not only unknown commands (it caught ~40 stale flags).
• doctor compares the stamped metadata.version, and also warns when an installed copy with the same version was edited.

Not in this PR

• init --browser|--api writes its inline starters; it doesn't fetch devhelmhq/templates. The templates PR mirrors init's layout. Fetching from templates would add a network dependency to an offline-safe command and needs a decision.
• @devhelm/synthetics exports (email.address, email.wait, webhook.wait) follow RFC A5.1. The package has no published API reference yet; the skill must be re-checked when it ships.
• CI on this branch inherits feat: synthetics surfaces for the CLI and devhelm.yml (doc 42 §6) #168's state (feat: synthetics surfaces for the CLI and devhelm.yml (doc 42 §6) #168 currently fails lint/typecheck/test in CI because @devhelm/sdk/bundler isn't resolvable from package.json). Locally everything passes with the sdk build installed.

How to verify locally

npm ci && npm run build
npm run skills:check
npx vitest run test/skills
node bin/run.js skills install --target=generic --path /tmp/x && ls /tmp/x/.skills

🤖 Generated with Claude Code