Add the Test Companion action

test-companion/README.md

@@ -0,0 +1,89 @@
+# test-companion
+
+Ask BrowserStack Test Companion for test work from a pull request.
+
+Comment `@test-companion test this PR` and it reads the change, proposes the
+tests it would write, and writes them once someone replies `go`. The result
+comes back as one comment and one check.
+
+## Quick start
+
+Two secrets — `BROWSERSTACK_USERNAME` and `BROWSERSTACK_ACCESS_KEY` — and the
+workflow below at `.github/workflows/test-companion.yml`. Copy it as-is; what
+runs is decided by `testcompanion-pipeline.yml` in your repository, not here.
+
+```yaml
+name: Test Companion
+
+on:
+    issue_comment:
+        types: [created]
+    pull_request:
+        types: [opened, reopened, synchronize, closed]
+
+jobs:
+    test-companion:
+        runs-on: ubuntu-latest
+        timeout-minutes: 35
+        permissions:
+            contents: write # branches and commits
+            pull-requests: write # the comment
+            checks: write # the "Test Companion" check
+            actions: write # stopping a run on request
+        steps:
+            - uses: actions/checkout@v4
+              with:
+                  fetch-depth: 0 # jobs that look at what changed need history
+
+            - uses: browserstack/github-actions/test-companion@v1
+              with:
+                  username: ${{ secrets.BROWSERSTACK_USERNAME }}
+                  access-key: ${{ secrets.BROWSERSTACK_ACCESS_KEY }}
+```
+
+## Pin the CLI version
+
+`cli-version` is optional and you should set it once you know which version you
+want.
+
+Without a pin the CLI upgrades itself the first time it runs on a release day.
+That spends your pipeline's time on a download, and it means two runs a day
+apart are not the same software.
+
+**Both lines or neither.** A version with no checksum is refused — "refusing an
+unverified pinned install" — and the job stops before it reports anything. Take
+both from the release you tested with.
+
+```yaml
+                  cli-version: "1.2.0"
+                  cli-sha256: "<sha256 of that release for linux-amd64>"
+```
+
+## Inputs
+
+| Input | Required | What it does |
+|---|---|---|
+| `username` | yes | BrowserStack username, from your secret store |
+| `access-key` | yes | BrowserStack access key, from your secret store |
+| `cli-version` | no | Pin the CLI. Strongly recommended — see above |
+| `cli-sha256` | no | Checksum for the pinned version |
+| `build-effect` | no | `off`, `report` or `fail` — whether a test failure fails your build |
+| `workflow` | no | Manual runs only; event-driven runs read the pipeline file |
+| `preview` | no | Report what would happen and change nothing |
+| `app-id` / `app-private-key` | no | Use the Test Companion GitHub App's identity |
+
+## What runs is not decided here
+
+This file is awkward to change once it is in many repositories, so it stays
+near-empty on purpose. Which jobs run, on which events, with which spending
+limits, lives in `testcompanion-pipeline.yml` in your own repository, which you
+can edit freely without touching the workflow above.
+
+## Notes for maintainers
+
+This is a **composite** action with **no dependencies** and no `dist/` bundle to
+rebuild — unlike the JavaScript actions elsewhere in this repository. The
+sources in `src/` are what run.
+
+Tests: `cd test-companion && node --test`. They need no network and no
+credentials.

test-companion/action.yml

@@ -0,0 +1,227 @@
+name: "Test Companion"
+description: >
+    Run Test Companion from a GitHub workflow — generate test cases, explain a red
+    build, fix stale tests — and report the result back onto the pull request.
+    Add this as one step with two secrets; everything else is read from the
+    testcompanion-pipeline.yml in your repository.
+author: "BrowserStack"
+branding:
+    icon: check-circle
+    color: orange
+
+inputs:
+    username:
+        description: "BrowserStack username. Store it as a repository secret, never inline."
+        required: true
+    access-key:
+        description: "BrowserStack access key. Store it as a repository secret, never inline."
+        required: true
+    app-id:
+        description: >-
+            Test Companion GitHub App id. With it, comments and the check come from
+            "Test Companion" rather than github-actions[bot], and the permissions
+            block below can be deleted. Optional — without it the workflow's own
+            token is used.
+        required: false
+    app-private-key:
+        description: >-
+            The App's private key. Store it as a repository or organisation secret,
+            never inline. It is never passed to the agent's shell.
+        required: false
+
+    workflow:
+        description: >
+            Which job to run, for a manual run only (see `tc workflow list`). Leave it
+            unset for event-driven runs, where the job comes from your pipeline file.
+        required: false
+    preview:
+        description: "Manual runs only: report what would happen and change nothing."
+        required: false
+        default: "false"
+
+    cli-version:
+        description: >
+            Exact Test Companion CLI version to install, e.g. 1.39.8. Pinning keeps
+            the same pipeline behaving the same next month. LEAVE IT UNSET AND THE
+            LATEST RELEASE IS INSTALLED, which can change between runs — so pin it
+            if you want a pipeline that does not move under you.
+        required: false
+    cli-sha256:
+        description: >
+            Checksum for the pinned CLI. Required alongside cli-version, because the
+            installer refuses an unverified pinned install.
+        required: false
+
+    config:
+        description: "Path to the pipeline file. Defaults to testcompanion-pipeline.yml at the repo root."
+        required: false
+    build-effect:
+        description: >
+            Whether a bad outcome reddens THIS step: off, report or fail. The
+            Test Companion check always reports truthfully regardless.
+        required: false
+        default: "report"
+    github-token:
+        description: >
+            Token used to post the check and the comment. Override only to post as a
+            different identity, such as a GitHub App.
+        required: false
+        default: ${{ github.token }}
+    self-workflow-name:
+        description: >
+            This workflow's own name, so a red run of it is never explained by itself.
+            Defaults to the running workflow.
+        required: false
+        default: ${{ github.workflow }}
+
+outputs:
+    outcome:
+        description: "One of: Succeeded, Tests failed, Run broke, Stopped at limit, Could not run."
+        value: ${{ steps.run.outputs.outcome }}
+    result-file:
+        description: >-
+            Path to the run's RESULT — the test cases it generated, with their steps.
+            Written whether or not they reached Test Management, which is the only
+            record a preview run leaves behind. Distinct from summary-file, which
+            records how the run went rather than what it produced.
+        value: ${{ steps.run.outputs.result-file }}
+    summary-file:
+        description: "Path to the run's report file."
+        value: ${{ steps.run.outputs.summary-file }}
+    cli-version:
+        description: "The CLI version that actually ran."
+        value: ${{ steps.install.outputs.cli-version }}
+
+runs:
+    using: composite
+    steps:
+        # A fresh runner has no git identity, and several jobs commit internally.
+        # Scoped to this repository rather than --global so nothing outlives the run.
+        - name: Give this run a git identity
+          shell: bash
+          run: |
+              set -euo pipefail
+              git config user.email "test-companion[bot]@users.noreply.github.com"
+              git config user.name "Test Companion"
+
+        - name: Install the Test Companion CLI
+          id: install
+          shell: bash
+          env:
+              TC_VERSION: ${{ inputs.cli-version }}
+              TC_SHA256: ${{ inputs.cli-sha256 }}
+          run: |
+              set -euo pipefail
+              # $RUNNER_TEMP, not /tmp: it is per-job and cleaned up, where a
+              # fixed /tmp path is shared with everything else on the runner
+              # and can be written before this step runs. The Action's own
+              # report files already take this care; this step did not.
+              installer="${RUNNER_TEMP:-/tmp}/tc-install.sh"
+              curl -fsSL -o "$installer" https://test-companion.browserstack.com/cli/install.sh
+              sh "$installer"
+              echo "$HOME/.local/bin" >> "$GITHUB_PATH"
+              version="$("$HOME/.local/bin/tc" --version 2>/dev/null | head -1 || echo unknown)"
+              # FR-2: pinning only means something if the pin actually took. The
+              # installer refuses an unverified pinned install on its own, but a
+              # silent fallback to latest would leave the run reporting a version
+              # nobody asked for — and "the same pipeline behaves the same next
+              # month" is the entire point of the input.
+              # EXACT, not a substring. `${version#*$TC_VERSION}` accepted a pin
+              # of 1.2.0 when 11.2.0 was installed, and a pin of 1.1 when 1.1.9
+              # was — so the check passed for a version nobody asked for, which
+              # is the one thing it exists to catch.
+              # The first version-shaped token. Stripping non-digits instead
+              # would fold the architecture into the number, turning a version
+              # line ending "linux/amd64" into 1.1.964.
+              installed="$(printf '%s' "$version" | grep -oE '[0-9]+(\.[0-9]+)+' | head -1 || true)"
+              if [ -n "${TC_VERSION:-}" ] && [ "$installed" != "$TC_VERSION" ]; then
+                echo "::error::Asked for Test Companion CLI $TC_VERSION but got '$version'." >&2
+                exit 1
+              fi
+              echo "cli-version=$version" >> "$GITHUB_OUTPUT"
+              echo "::notice::Test Companion $version"
+
+        # The CLI only picks up ambient BROWSERSTACK_* credentials during `auth
+        # login`; other commands just find no stored session and exit 3. Assuming
+        # the environment variables alone were enough cost a live run to discover.
+        - name: Sign in
+          shell: bash
+          env:
+              BROWSERSTACK_USERNAME: ${{ inputs.username }}
+              BROWSERSTACK_ACCESS_KEY: ${{ inputs.access-key }}
+              # FR-2. This is the FIRST tc command of the job, and without the
+              # pin here the CLI self-upgrades during `auth login` — before the
+              # Run step below ever applies it. The pin then correctly reports
+              # a mismatch against the binary the pin was meant to preserve,
+              # so ASKING FOR A PINNED VERSION MADE THE RUN FAIL. Seen live on
+              # 2026-10-29 when tc 1.1.9 shipped: "pinned to version 1.1.8 but
+              # the installed binary is 1.1.9".
+              #
+              # Every step that runs tc needs this, not just the one that does
+              # the work.
+              TC_PIN_VERSION: ${{ inputs.cli-version }}
+              TC_SKIP_AGENT_SKILL_INSTALL: "1"
+              TC_ACTION_CLI_VERSION: ${{ steps.install.outputs.cli-version }}
+          run: |
+              set -euo pipefail
+              # Bounded: a surface that cannot draw a browser must never sit waiting
+              # on an interactive sign-in.
+              timeout 120 tc auth login
+
+        - name: Run
+          id: run
+          shell: bash
+          env:
+              # The CLI picks these up on a non-interactive shell with no extra flags.
+              BROWSERSTACK_USERNAME: ${{ inputs.username }}
+              BROWSERSTACK_ACCESS_KEY: ${{ inputs.access-key }}
+              # Stops the CLI upgrading itself mid-pipeline, so a pinned version stays
+              # pinned and the 90-second first-response budget is not spent on a download.
+              TC_PIN_VERSION: ${{ inputs.cli-version }}
+              # Only this step may talk to the GitHub API. Deliberately not exported to
+              # the environment the agent's own shell commands run in.
+              GH_TOKEN: ${{ inputs.github-token }}
+              # A CI runner has no Claude Code to install a skill into; it just adds
+              # noise and a write to the home directory.
+              TC_SKIP_AGENT_SKILL_INSTALL: "1"
+              TC_ACTION_WORKFLOW: ${{ inputs.workflow }}
+              TC_ACTION_PREVIEW: ${{ inputs.preview }}
+              TC_ACTION_CONFIG: ${{ inputs.config }}
+              TC_ACTION_BUILD_EFFECT: ${{ inputs.build-effect }}
+              TC_ACTION_SELF_WORKFLOW: ${{ inputs.self-workflow-name }}
+              TC_ACTION_APP_ID: ${{ inputs.app-id }}
+              TC_ACTION_APP_PRIVATE_KEY: ${{ inputs.app-private-key }}
+          run: node "${{ github.action_path }}/src/main.mjs"
+
+        - name: Attach the report
+          if: always() && steps.run.outputs.summary-file != ''
+          # v4.6.2, pinned to a commit: a tag can be repointed by its owner,
+          # and this step runs in the customer's job with the customer's token.
+          uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02
+          with:
+              name: test-companion-report
+              path: ${{ steps.run.outputs.summary-file }}
+              if-no-files-found: ignore
+
+        # Separate artifact from the report, because it answers a different
+        # question and has a different reader: the report says how the run went,
+        # this is the work itself. For a preview run it is the only place that
+        # work survives, so it is attached even when the run is about to go red.
+        - name: Attach the generated test cases
+          if: always() && steps.run.outputs.result-file != ''
+          # v4.6.2, pinned to a commit: a tag can be repointed by its owner,
+          # and this step runs in the customer's job with the customer's token.
+          uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02
+          with:
+              name: test-companion-result
+              path: ${{ steps.run.outputs.result-file }}
+              if-no-files-found: ignore
+
+        # Separate from the run step so the report is attached even when the build
+        # effect says this run should redden the build.
+        - name: Apply the build effect
+          if: always() && steps.run.outputs.should-fail == 'true'
+          shell: bash
+          run: |
+              echo "::error::Test Companion: ${{ steps.run.outputs.outcome }}"
+              exit 1