Skip to content

test: keep the developer's gateway key out of the test environment - #11

Merged
thymikee merged 1 commit into
mainfrom
t3/hermetic-test-env
Oct 14, 2026
Merged

thymikee merged 1 commit into
mainfrom
t3/hermetic-test-env

Conversation

@thymikee

Copy link
Copy Markdown
Member

npm test failed locally on main (10 tests) when AI_GATEWAY_API_KEY is exported. The test fixture starts from process.env and sets CALLSTACK_AUTH_TOKEN, so with both keys present defaultProvider() picks Vercel AI Gateway and every test that expects the Callstack portal fails. CI never saw it because it has no keys.

  • The fixture now removes every gateway key except the Callstack one it sets.
  • It also removes XDG_DATA_HOME, so OpenCode's key-store lookup can't reach the developer's real auth.json.

npm test: 65/65 pass both with and without AI_GATEWAY_API_KEY / XDG_DATA_HOME set.

🤖 Generated with Claude Code

The fixture copied process.env, so an exported AI_GATEWAY_API_KEY made the
default gateway Vercel and broke every test expecting the Callstack portal.
Drop non-Callstack gateway keys and XDG_DATA_HOME (OpenCode's key store) too.
@thymikee
thymikee merged commit bace670 into main Oct 14, 2026
2 checks passed
thymikee pushed a commit that referenced this pull request Oct 20, 2026
…version (#13)

* fix: ask the OpenCode binary for --standalone instead of reading its version

The version probe decided the launch from `--version`, but every dev, beta
and source build of *both* OpenCode generations reports `0.0.0-*` (or
`local`), which the regex never matched, so they all fell back to the v1
launch. Verified against the published `@opencode/cli@beta`:

    $ opencode --version
    opencode v0.0.0-beta-19507       <- probe: no match -> major 1
    $ apex run opencode
    Unrecognized flag: --model in command opencode

The whole preview channel hits the exact error PR #12 exists to prevent,
and `0.0.0-*` cannot be classified by version at all: the old `opencode-ai`
package has published over 11k `0.0.0-*` versions too, so "major 0 means
v2" would just move the bug to v1 dev builds.

Probe the capability instead. `--standalone` exists if and only if the
interface is v2, and `--help` says so on all four real binaries, including
the `0.0.0-*` pair no regex can separate: v1 stable, v1 dev, v2 beta and
v2 stable all answer correctly. v1 writes its help to stderr and v2 to
stdout, so both streams are read.

The probe now answers a tri-state: true (OpenCode 2), false (OpenCode 1),
or null (no answer). It never rejects: execFile can throw synchronously (a
.cmd launcher since CVE-2024-27980) and a failed or timed-out --help is no
answer either. On null, `apex run` adds neither --model nor --standalone,
so the assistant still starts on the user's own config, and says so on
stderr rather than guessing a generation and dying on its flag; the init
summary admits the same instead of blindly showing the v1 expansion.

Also scrub OPENCODE_CONFIG_CONTENT in the test fixture: it reached launched
fakes from the developer's shell, the same leak #11 closed for gateway keys.

Docs-only inaccuracies PR #12 merged (the "flags only follow a subcommand"
comment, the `auth list` example) are untouched here and tracked as F6.

* refactor: simplify the probe and its messages per the review of this PR

The fanout review found no bugs; these are its confirmed findings.

- openCodeSupportsStandalone becomes a short async function over
  promisify(execFile): one try/catch covers the callback's error branch
  and what execFile throws synchronously (D2).
- A failing or timed-out --help that printed --standalone is an answer
  after all; the captured streams on the error are read instead of
  thrown away (D1).
- launchOptions owns the unknown-probe sentence as `warning` and says
  "interface", not "version"; commandRun prints it instead of writing
  its own wording of the same state (D6, D9).
- The init summary drops the note when no binary was found: nothing was
  asked, so nothing is unknown (D5).
- runExpansion joins the command tokens instead of repairing a trailing
  space with trimEnd() (D10).
- Tests: the synchronous-throw case uses a path that actually reaches
  execFile (a NUL byte); the probe asserts a failed --help with and
  without the flag in its output; the CLI fake sheds the --version
  plumbing Apex no longer reads and the stderr cases the probe unit
  test already covers (D3, D8).

* chore: tighten comments around the OpenCode probe
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant