From 09e83d77f0f2d9504869a9b8426f2b608fb19ceb Mon Sep 17 00:00:00 2001 From: Changyong Gong Date: Fri, 9 Oct 2026 14:00:21 +0800 Subject: [PATCH 1/5] Centralize Java team-memory coordination in Java Pack Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- .github/actions/queue-team-memory/README.md | 146 +++++ .github/actions/queue-team-memory/action.yml | 46 ++ .../actions/queue-team-memory/team_memory.py | 348 ++++++++++ .../tests/test_team_memory.py | 619 ++++++++++++++++++ .github/workflows/team-memory-coordinator.yml | 137 ++++ .github/workflows/team-memory-post-merge.yml | 44 +- .github/workflows/team-memory-tests.yml | 39 ++ 7 files changed, 1351 insertions(+), 28 deletions(-) create mode 100644 .github/actions/queue-team-memory/README.md create mode 100644 .github/actions/queue-team-memory/action.yml create mode 100644 .github/actions/queue-team-memory/team_memory.py create mode 100644 .github/actions/queue-team-memory/tests/test_team_memory.py create mode 100644 .github/workflows/team-memory-coordinator.yml create mode 100644 .github/workflows/team-memory-tests.yml diff --git a/.github/actions/queue-team-memory/README.md b/.github/actions/queue-team-memory/README.md new file mode 100644 index 00000000..6bd6e48c --- /dev/null +++ b/.github/actions/queue-team-memory/README.md @@ -0,0 +1,146 @@ +# Java team-memory coordinator + +Java Pack hosts the single queue for the `microsoft/vscode-java-pack` wiki. +The [push caller](../../workflows/team-memory-post-merge.yml) only preserves +source identities and dispatches the [coordinator](../../workflows/team-memory-coordinator.yml). +Manual reconciliation now runs through the coordinator too, not the old push +workflow. No source caller logs in to Azure or invokes the agent. + +Both workflows require **`ISSUELENS_TEAM_MEMORY_COORDINATOR_ENABLED=true`**. +This is a new, disabled-by-default repository variable: the existing +`ISSUELENS_TEAM_MEMORY_ENABLED` does not enable this rollout. This PR does not +change variables, credentials, App installations, Azure access, or agent +deployments. Verify the prerequisites below before a separately authorized +enablement. Issue-loop and direct-task consumers are unchanged. + +## Source trust and scope + +The reviewed allowlist in `team_memory.py` binds names, numeric repository IDs, +public visibility, and default branches. These sources currently configure +`instructions.team_memory.wiki_repository: microsoft/vscode-java-pack`: + +| Source | Default branch | Reviewed policy snapshot | +| --- | --- | --- | +| `microsoft/vscode-java-pack` | `main` | [0823dbde](https://github.com/microsoft/vscode-java-pack/blob/0823dbde1947de111c39890fc095f55c0ff0d258/.github/issuelens.yml) | +| `microsoft/vscode-java-debug` | `main` | [3705df0d](https://github.com/microsoft/vscode-java-debug/blob/3705df0d0b1215b044e759647e50e4caa231e5b6/.github/issuelens.yml) | +| `microsoft/vscode-java-test` | `main` | [cff6fdb4](https://github.com/microsoft/vscode-java-test/blob/cff6fdb4b5e3221a4e88473d2f90e1fd58f1061c/.github/issuelens.yml) | +| `microsoft/vscode-java-dependency` | `main` | [eb3fba04](https://github.com/microsoft/vscode-java-dependency/blob/eb3fba04f2763416b25833d92df4fb0210a9d668/.github/issuelens.yml) | +| `microsoft/vscode-maven` | `main` | [eee9eb0b](https://github.com/microsoft/vscode-maven/blob/eee9eb0b49b8b5bdd4a479646d700340a2b37cbd/.github/issuelens.yml) | +| `microsoft/vscode-gradle` | `develop` | [1a544c91](https://github.com/microsoft/vscode-gradle/blob/1a544c917399ead6c23251301a041e505c0a6926/.github/issuelens.yml) | +| `microsoft/java-debug` | `main` | [bcae605a](https://github.com/microsoft/java-debug/blob/bcae605a6240f11b447163a614e215721c11b107/.github/issuelens.yml) | +| `microsoft/build-server-for-gradle` | `develop` | [ab37cca9](https://github.com/microsoft/build-server-for-gradle/blob/ab37cca97693c60e90d257ce9eb6e9575da7e6f1/.github/issuelens.yml) | + +The two Spring repositories are not included: they currently have no +`.github/issuelens.yml` policy. An allowlist entry is not a source migration or +authorization to ignore policy. Only Java Pack's caller changes in this PR. +Other repositories must adopt this action in separate reviewed PRs. +Renames, transfers, branch changes, or private sources require another review. + +The hosted agent must resolve each source's policy and privacy constraints, +verify that its validated destination is this wiki, and stop rather than +override a conflicting policy. Requests explicitly require this destination; +the pinned client's final receipt validator rejects any other wiki. Neither +the dispatch action nor its allowlist grants wiki write access. + +## Future external callers + +Use `.github/workflows/team-memory-post-merge.yml` on the source's **actual +default branch**, gated by its own opt-in variable, with job permissions +`contents: read` and `actions: write`. No source checkout or copied Python is +needed for a remote reference: + +```yaml +- name: Queue Java team-memory update + uses: microsoft/vscode-java-pack/.github/actions/queue-team-memory@FULL_REVIEWED_COMMIT_SHA + with: + source-token: ${{ github.token }} + dispatch-token: ${{ steps.dispatch-token.outputs.token }} +``` + +Replace the placeholder with the reviewed, merged 40-character revision of +this action; do not use a moving branch. The caller must obtain +`steps.dispatch-token.outputs.token` using a **dedicated dispatch GitHub App**, +restricted to Java Pack with **Actions write**. The read and dispatch tokens +are separate inputs. An explicitly empty token fails; it never falls back. +Do not distribute the hosted IssueLens App's private key to source workflows. +A source repository's `GITHUB_TOKEN` remains repository-scoped regardless of +its `actions: write` permission. Java Pack's own caller can use its token for +both inputs without new credentials. + +## Coordinator prerequisites + +- Existing central Azure OIDC configuration and secrets: `AZURE_CLIENT_ID`, + `AZURE_TENANT_ID`, `AZURE_SUBSCRIPTION_ID`, `ISSUELENS_AGENT_URL`, and + `ISSUELENS_AGENT_SCOPE`. Their existing names are preserved; secret presence + does not establish that the identity or endpoint is ready. +- For external sources, a separate read-only GitHub App installation covering + the selected allowlisted source, with **Actions read, Contents read, and + Pull requests read**. Configure central secrets + `ISSUELENS_SOURCE_READ_APP_CLIENT_ID` and + `ISSUELENS_SOURCE_READ_APP_PRIVATE_KEY`. The workflow mints an ephemeral, + single-source token only after validating source selection and revokes it at + job end. Java Pack requests use its own `GITHUB_TOKEN` instead. Source-read + and dispatch installations may differ and must mint separate tokens. +- The existing hosted IssueLens identity must independently have permission + to read the source and publish to the Java Pack wiki, with its normal + policy/privacy and wiki snapshot-precondition enforcement. Do not broaden + its source-code/issue write permissions for this feature. + +The new App secrets are setup prerequisites, not assumed existing credentials. +No live invocation is required to validate these repository changes. + +## Verification and outcomes + +Automatic dispatch requires explicit `source_repository`, `source_run_id`, +`source_run_attempt`, and `source_artifact_id`. Manual dispatch supplies +`source_repository` and one `pull_request_number`, with all three automatic +identifiers empty. Mixed or incomplete requests fail before source access. +All runs use the same fixed wiki concurrency group, `queue: max`, and +`cancel-in-progress: false`, including validation, discovery, invocation, and +final receipt verification. GitHub supports one active run and at most 100 +pending runs in this queue; it is not an unlimited or guaranteed-delivery queue. +See GitHub's [concurrency documentation](https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/control-workflow-concurrency) +for the `queue: max` contract. + +The dispatcher accepts only a trusted, non-created/non-deleted/non-forced +default-branch push from the exact source workflow. Its artifact contains +repository/run/workflow identities, before/after SHAs, and commit IDs only: +no commit messages, bodies, code, credentials, or agent output. It is capped +at 64 KiB and 1,000 commits and retained for seven days. + +Before downloading, the coordinator authenticates source run/attempt and +artifact metadata, repository IDs, workflow, branch, actors, digest, expiry, +and size. Pinned artifact download enforces the digest. Preflight repeats the +metadata checks, validates the downloaded identity-only schema, and reuses +IssueLens's complete bounded push/PR discovery and merge revalidation. The +client is pinned to the actual merged revision of +[microsoft/IssueLens#50](https://github.com/microsoft/IssueLens/pull/50), +`4175ea71e170938826fb847e6bd5108f0f5597cf`. Only the small Java provenance +adapter is local; request, stream, discovery, and receipt machinery is not +copied or monkey-patched. The pilot's IssueLens-only coordinator adapter is +deliberately not called. + +Requests preserve source and coordinator identities separately, require +current-source/wiki comparison, and authorize wiki knowledge writes only. +A successful SSE stream or accepted dispatch is **not successful publication**. +The pinned client validates every PR outcome and the wiki identity/SHA; +partial, failed, needs-review, missing, or wrong-wiki receipts fail the job. +No merged PRs is an explicit preflight skip before Azure login. + +There is one dispatch POST and one agent submission, without automatic retries. +An uploaded artifact does not prove accepted dispatch. On ambiguous dispatch, +timeout, or agent/write outcome, inspect coordinator runs and confirmed wiki +state before manually reconciling through this queue. It does not serialize +chat/direct/external writers or prove a timed-out hosted invocation stopped. + +## Offline checks + +With the pinned IssueLens action files available locally, set +`ISSUELENS_CLIENT_PATH` to their `issuelens_action.py` and run: + +```text +python -m unittest discover -s .github/actions/queue-team-memory/tests +``` + +The focused CI workflow loads the same immutable client revision. These checks +use fake API/stream responses and make no dispatch, Azure, or wiki calls. diff --git a/.github/actions/queue-team-memory/action.yml b/.github/actions/queue-team-memory/action.yml new file mode 100644 index 00000000..07ca85e8 --- /dev/null +++ b/.github/actions/queue-team-memory/action.yml @@ -0,0 +1,46 @@ +name: Queue Java team memory +description: Preserve an allowlisted default-branch push and dispatch the central Java wiki coordinator. + +inputs: + source-token: + description: Source repository read token for push provenance validation. + required: false + default: ${{ github.token }} + dispatch-token: + description: Token with Actions write access to microsoft/vscode-java-pack. + required: false + default: ${{ github.token }} + +outputs: + source-artifact-id: + description: Uploaded identity artifact ID; not proof of dispatch acceptance or successful maintenance. + value: ${{ steps.artifact.outputs.artifact-id }} + +runs: + using: composite + steps: + - name: Prepare identity-only source event + id: source + shell: bash + env: + GITHUB_ACTION_PATH: ${{ github.action_path }} + GH_TOKEN: ${{ inputs.source-token }} + run: python3 -I "$GITHUB_ACTION_PATH/team_memory.py" prepare-dispatch + + - name: Preserve source event for the coordinator + id: artifact + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: issuelens-team-memory-source-${{ github.run_attempt }} + path: ${{ steps.source.outputs.source-event-path }} + if-no-files-found: error + retention-days: 7 + + - name: Queue one team-memory request + shell: bash + env: + GITHUB_ACTION_PATH: ${{ github.action_path }} + GH_TOKEN: ${{ inputs.source-token }} + DISPATCH_TOKEN: ${{ inputs.dispatch-token }} + SOURCE_ARTIFACT_ID: ${{ steps.artifact.outputs.artifact-id }} + run: python3 -I "$GITHUB_ACTION_PATH/team_memory.py" dispatch diff --git a/.github/actions/queue-team-memory/team_memory.py b/.github/actions/queue-team-memory/team_memory.py new file mode 100644 index 00000000..78888a5c --- /dev/null +++ b/.github/actions/queue-team-memory/team_memory.py @@ -0,0 +1,348 @@ +"""Identity-only Java dispatch and cross-repository adapter for the pinned IssueLens client.""" + +import argparse +import importlib.util +import json +import os +import re +import tempfile +import urllib.request +from datetime import datetime, timedelta, timezone +from pathlib import Path + +COORDINATOR = "microsoft/vscode-java-pack" +SOURCE_WORKFLOW = ".github/workflows/team-memory-post-merge.yml" +COORDINATOR_WORKFLOW = ".github/workflows/team-memory-coordinator.yml" +SOURCE_INPUTS = ("SOURCE_RUN_ID", "SOURCE_RUN_ATTEMPT", "SOURCE_ARTIFACT_ID") +MAX_SOURCE_BYTES = 64 * 1024 +MAX_PUSH_COMMITS = 1000 +# Reviewed public sources with an existing policy targeting the Java Pack wiki. +SOURCES = { + COORDINATOR: (104967329, "main"), + "microsoft/vscode-java-debug": (102584737, "main"), + "microsoft/vscode-java-test": (110522074, "main"), + "microsoft/vscode-java-dependency": (129053101, "main"), + "microsoft/vscode-maven": (116921700, "main"), + "microsoft/vscode-gradle": (216314492, "develop"), + "microsoft/java-debug": (102583752, "main"), + "microsoft/build-server-for-gradle": (626812412, "develop"), +} + + +def require(condition, message): + if not condition: + raise ValueError(message) + + +def positive(value): + require(re.fullmatch(r"[1-9][0-9]{0,14}", str(value)), "Invalid positive numeric identifier") + return int(value) + + +def full_sha(value): + require(isinstance(value, str) and re.fullmatch(r"[0-9a-f]{40}", value) + and value != "0" * 40, "Expected a full nonzero commit SHA") + return value + + +def unique_object(pairs): + result = {} + for key, value in pairs: + require(key not in result, "Duplicate JSON key") + result[key] = value + return result + + +def read_json(path, limit): + with path.open("rb") as source: + content = source.read(limit + 1) + require(len(content) <= limit, "Source JSON exceeds its size limit") + result = json.loads(content, object_pairs_hook=unique_object) + require(isinstance(result, dict), "Expected a JSON object") + return result + + +def matching_metadata(actual, expected): + return (isinstance(actual, dict) and actual == expected + and all(type(actual[key]) is type(value) for key, value in expected.items())) + + +class NoRedirect(urllib.request.HTTPRedirectHandler): + def redirect_request(self, *args, **kwargs): + return None + + +def github_request(path, token, payload=None): + require(isinstance(token, str) and token.strip(), "Required GitHub token is empty") + return urllib.request.Request( + "https://api.github.com" + path, + data=None if payload is None else json.dumps(payload).encode("utf-8"), + headers={"Authorization": "Bearer " + token, "Accept": "application/vnd.github+json", + "Content-Type": "application/json", "X-GitHub-Api-Version": "2022-11-28"}, + ) + + +def github_read(path, token): + with urllib.request.build_opener(NoRedirect()).open(github_request(path, token), timeout=30) as response: + content = response.read(4 * 1024 * 1024 + 1) + require(len(content) <= 4 * 1024 * 1024, "GitHub response exceeds its size limit") + result = json.loads(content, object_pairs_hook=unique_object) + require(isinstance(result, dict), "Invalid GitHub response") + return result + + +def project(repository, token): + require(repository in SOURCES, "Source repository is not allowlisted") + identity, branch = SOURCES[repository] + result = github_read(f"/repos/{repository}", token) + require(result.get("full_name") == repository and result.get("id") == identity + and result.get("default_branch") == branch + and result.get("private") is False and result.get("archived") is False, + "Repository identity, public visibility, or default branch changed; review the allowlist") + return result + + +def workflow_context(repository, workflow, event_name, token): + require(os.environ["GITHUB_REPOSITORY"] == repository + and os.environ["GITHUB_EVENT_NAME"] == event_name, "Unexpected workflow context") + event = read_json(Path(os.environ["GITHUB_EVENT_PATH"]), 32 * 1024 * 1024) + current = project(repository, token) + require(isinstance(event.get("repository"), dict) + and event["repository"].get("full_name") == repository + and event["repository"].get("id") == current["id"], "Event repository mismatch") + reference = "refs/heads/" + current["default_branch"] + require(os.environ["GITHUB_REF"] == reference + and os.environ["GITHUB_WORKFLOW_REF"] == repository + "/" + workflow + "@" + reference, + "Run the trusted workflow from the current default branch") + full_sha(os.environ["GITHUB_WORKFLOW_SHA"]) + return current, event + + +def validate_push(event, reference, head): + require(all(event.get(flag) is False for flag in ("created", "deleted", "forced")), + "Created, deleted, or forced refs require manual PR reconciliation") + before, after = full_sha(event.get("before")), full_sha(event.get("after")) + require(before != after and event.get("ref") == reference and head == after + and isinstance(event.get("head_commit"), dict) and event["head_commit"].get("id") == after, + "Push source identity mismatch") + commits = event.get("commits") + require(isinstance(commits, list) and 0 < len(commits) <= MAX_PUSH_COMMITS + and all(isinstance(item, dict) for item in commits), "Missing or oversized push inventory") + shas = [full_sha(item.get("id")) for item in commits] + require(len(set(shas)) == len(shas) and after in shas and before not in shas, + "Push inventory has duplicate or mismatched identities") + return before, after, shas + + +def source_event_path(): + return Path(os.environ["RUNNER_TEMP"]) / "issuelens-team-memory-source" / "source-event.json" + + +def outputs(values): + with open(os.environ["GITHUB_OUTPUT"], "a", encoding="utf-8") as output: + for name, value in values.items(): + output.write(f"{name}={value}\n") + + +def push_snapshot(): + repository = os.environ["GITHUB_REPOSITORY"] + current, event = workflow_context(repository, SOURCE_WORKFLOW, "push", os.environ["GH_TOKEN"]) + before, after, shas = validate_push(event, os.environ["GITHUB_REF"], os.environ["GITHUB_SHA"]) + require(os.environ["GITHUB_WORKFLOW_SHA"] == after, "Source workflow revision does not match the push") + actors = [os.environ[name] for name in ("GITHUB_ACTOR", "GITHUB_TRIGGERING_ACTOR")] + require(all(re.fullmatch(r"[A-Za-z0-9-]+(?:\[bot\])?", actor) for actor in actors), "Invalid source actors") + return { + "metadata": { + "repository": repository, "repository_id": current["id"], "base_ref": current["default_branch"], + "event_name": "push", "event_action": "push", + "actor_login": actors[0], "triggering_actor": actors[1], + "workflow_ref": os.environ["GITHUB_WORKFLOW_REF"], "workflow_sha": after, + "run_id": positive(os.environ["GITHUB_RUN_ID"]), "run_attempt": positive(os.environ["GITHUB_RUN_ATTEMPT"]), + }, + "event": { + "repository": {"id": current["id"], "full_name": repository}, "ref": event["ref"], + "before": before, "after": after, "created": False, "deleted": False, "forced": False, + "commits": [{"id": sha} for sha in shas], "head_commit": {"id": after}, + }, + } + + +def prepare_dispatch(): + snapshot = push_snapshot() + content = json.dumps(snapshot, separators=(",", ":")).encode("utf-8") + require(len(content) <= MAX_SOURCE_BYTES, "Source event exceeds 64 KiB") + path = source_event_path() + path.parent.mkdir(exist_ok=True) + path.write_bytes(content) + outputs({"source-event-path": path}) + print("Prepared identity-only team-memory source event") + + +def dispatch(): + token = os.environ["DISPATCH_TOKEN"] + require(token.strip(), "dispatch-token must be non-empty") + snapshot = read_json(source_event_path(), MAX_SOURCE_BYTES) + expected = push_snapshot() + require(snapshot == expected and matching_metadata(snapshot["metadata"], expected["metadata"]), + "Source event changed before dispatch") + central = project(COORDINATOR, token) + metadata = snapshot["metadata"] + request = github_request( + f"/repos/{COORDINATOR}/actions/workflows/team-memory-coordinator.yml/dispatches", token, + {"ref": central["default_branch"], "inputs": { + "source_repository": metadata["repository"], "source_run_id": str(metadata["run_id"]), + "source_run_attempt": str(metadata["run_attempt"]), + "source_artifact_id": str(positive(os.environ["SOURCE_ARTIFACT_ID"])), + }}, + ) + with urllib.request.build_opener(NoRedirect()).open(request, timeout=30) as response: + require(response.status in {200, 204}, "Dispatch was not acknowledged; do not retry blindly") + print("Coordinator dispatch accepted; maintenance completion is reported by the coordinator run") + + +def selection(): + repository = os.environ.get("SOURCE_REPOSITORY", "") + require(repository in SOURCES, "Supply an allowlisted source_repository") + values = [os.environ.get(name, "") for name in SOURCE_INPUTS] + manual = os.environ.get("DISPATCH_PR", "") + require((all(values) and not manual) or (not any(values) and manual), + "Supply all three source identifiers or one manual PR, not both") + identifiers = tuple(positive(value) for value in values) if all(values) else None + if manual: + positive(manual) + workflow_context(COORDINATOR, COORDINATOR_WORKFLOW, "workflow_dispatch", os.environ["COORDINATOR_TOKEN"]) + return repository, identifiers + + +def select_source(): + repository, _ = selection() + outputs({"source-name": repository.split("/")[1], + "external": "true" if repository != COORDINATOR else "false"}) + + +def verify_source(repository, current, identifiers): + run_id, attempt, artifact_id = identifiers + token = os.environ["GH_TOKEN"] + run = github_read(f"/repos/{repository}/actions/runs/{run_id}/attempts/{attempt}", token) + require(type(run.get("id")) is int and run["id"] == run_id + and type(run.get("run_attempt")) is int and run["run_attempt"] == attempt and run.get("event") == "push" + and run.get("path") == SOURCE_WORKFLOW and run.get("head_branch") == current["default_branch"] + and isinstance(run.get("repository"), dict) and run["repository"].get("id") == current["id"] + and run["repository"].get("full_name") == repository + and isinstance(run.get("head_repository"), dict) and run["head_repository"].get("id") == current["id"] + and run["head_repository"].get("full_name") == repository, + "Source run is not the allowlisted default-branch push workflow") + head = full_sha(run.get("head_sha")) + actors = [run.get(name) for name in ("actor", "triggering_actor")] + require(all(isinstance(actor, dict) and isinstance(actor.get("login"), str) + and re.fullmatch(r"[A-Za-z0-9-]+(?:\[bot\])?", actor["login"]) for actor in actors), + "Invalid source run actors") + artifact = github_read(f"/repos/{repository}/actions/artifacts/{artifact_id}", token) + origin = artifact.get("workflow_run") + require(artifact.get("id") == artifact_id and artifact.get("expired") is False + and artifact.get("name") == f"issuelens-team-memory-source-{attempt}" + and isinstance(artifact.get("digest"), str) + and re.fullmatch(r"sha256:[0-9a-f]{64}", artifact["digest"]) + and type(artifact.get("size_in_bytes")) is int and 0 < artifact["size_in_bytes"] <= MAX_SOURCE_BYTES + and isinstance(origin, dict) and origin.get("id") == run_id + and origin.get("repository_id") == current["id"] and origin.get("head_repository_id") == current["id"] + and origin.get("head_branch") == current["default_branch"] and origin.get("head_sha") == head, + "Source artifact is expired, oversized, lacks a digest, or belongs to a different run") + created = datetime.fromisoformat(artifact["created_at"].replace("Z", "+00:00")) + expires = datetime.fromisoformat(artifact["expires_at"].replace("Z", "+00:00")) + require(created.tzinfo is not None and expires.tzinfo is not None + and created < expires <= created + timedelta(days=7) + and created <= datetime.now(timezone.utc) < expires, "Source artifact is outside its seven-day lifetime") + return { + "repository": repository, "repository_id": current["id"], "base_ref": current["default_branch"], + "event_name": "push", "event_action": "push", + "actor_login": actors[0]["login"], "triggering_actor": actors[1]["login"], + "workflow_ref": repository + "/" + SOURCE_WORKFLOW + "@refs/heads/" + current["default_branch"], + "workflow_sha": head, "run_id": run_id, "run_attempt": attempt, + } + + +def validate_dispatch(): + repository, identifiers = selection() + current = project(repository, os.environ["GH_TOKEN"]) + if identifiers is not None: + verify_source(repository, current, identifiers) + outputs({"automatic": "true" if identifiers is not None else "false", "source-repository": repository, + **({"source-run-id": identifiers[0], "source-artifact-id": identifiers[2]} if identifiers else {})}) + print("Validated queued source metadata" if identifiers else "Validated manual source selection") + + +def load_client(): + spec = importlib.util.spec_from_file_location("issuelens_client", os.environ["ISSUELENS_CLIENT_PATH"]) + require(spec is not None and spec.loader is not None, "Pinned IssueLens client is unavailable") + client = importlib.util.module_from_spec(spec) + spec.loader.exec_module(client) + return client + + +def coordinator_metadata(): + return { + "coordinator_repository": COORDINATOR, "coordinator_workflow_ref": os.environ["GITHUB_WORKFLOW_REF"], + "coordinator_workflow_sha": full_sha(os.environ["GITHUB_WORKFLOW_SHA"]), + "coordinator_run_id": positive(os.environ["GITHUB_RUN_ID"]), + "coordinator_run_attempt": positive(os.environ["GITHUB_RUN_ATTEMPT"]), + "required_wiki_repository": COORDINATOR, + } + + +def preflight(): + repository, identifiers = selection() + current = project(repository, os.environ["GH_TOKEN"]) + central = coordinator_metadata() + client = load_client() + client.display_options() + if identifiers is not None: + metadata = verify_source(repository, current, identifiers) + snapshot = read_json(source_event_path(), MAX_SOURCE_BYTES) + require(set(snapshot) == {"metadata", "event"} and matching_metadata(snapshot["metadata"], metadata), + "Source artifact metadata does not match the verified run") + push = snapshot["event"] + require(isinstance(push, dict) + and set(push) == {"repository", "ref", "before", "after", "created", "deleted", + "forced", "commits", "head_commit"} + and push["repository"] == {"id": current["id"], "full_name": repository} + and isinstance(push["commits"], list) + and all(isinstance(item, dict) and set(item) == {"id"} for item in push["commits"]) + and push["head_commit"] == {"id": metadata["workflow_sha"]}, + "Invalid identity-only push artifact") + validate_push(push, "refs/heads/" + current["default_branch"], metadata["workflow_sha"]) + try: + envelope = client.prepare_push_memory(repository, current, push, {**metadata, **central}) + except client.SkippedRequest as skipped: + outputs({"eligible": "false", "status": "skipped", "skip-reason": str(skipped)}) + print(f"No maintenance request: {skipped}") + return + else: + merged = client.read_merged_pr(repository, current, positive(os.environ["DISPATCH_PR"])) + metadata = {"repository": repository, "repository_id": current["id"], "base_ref": current["default_branch"], + "event_name": "workflow_dispatch", "event_action": "workflow_dispatch", **merged, **central} + envelope = {"metadata": metadata, "request": client.build_team_memory_request(metadata)} + envelope["request_type"] = "team-memory" + descriptor, path = tempfile.mkstemp(prefix="issuelens-request-", suffix=".json", dir=os.environ["RUNNER_TEMP"]) + with os.fdopen(descriptor, "w", encoding="utf-8") as request: + json.dump(envelope, request) + outputs({"request-path": path, "eligible": "true"}) + print("Prepared bounded Java wiki maintenance request") + + +def run(command): + try: + {"prepare-dispatch": prepare_dispatch, "dispatch": dispatch, "select-source": select_source, + "validate-dispatch": validate_dispatch, "preflight": preflight}[command]() + except ValueError as error: + raise SystemExit(f"::error::{error}") from None + except Exception: + message = ("Coordinator dispatch failed or its outcome is unknown; inspect coordinator runs before retrying" + if command == "dispatch" else "Java team-memory validation failed; no agent request was sent") + raise SystemExit(f"::error::{message}") from None + + +if __name__ == "__main__": + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("command", choices=("prepare-dispatch", "dispatch", "select-source", "validate-dispatch", "preflight")) + run(parser.parse_args().command) diff --git a/.github/actions/queue-team-memory/tests/test_team_memory.py b/.github/actions/queue-team-memory/tests/test_team_memory.py new file mode 100644 index 00000000..02c4165c --- /dev/null +++ b/.github/actions/queue-team-memory/tests/test_team_memory.py @@ -0,0 +1,619 @@ +import copy +import importlib.util +import io +import json +import os +import tempfile +import unittest +from datetime import datetime, timedelta, timezone +from email.message import Message +from pathlib import Path +from unittest.mock import Mock, patch + +ACTION = Path(__file__).resolve().parents[1] +ROOT = ACTION.parents[2] +spec = importlib.util.spec_from_file_location("java_team_memory", ACTION / "team_memory.py") +action = importlib.util.module_from_spec(spec) +spec.loader.exec_module(action) + + +class Response: + def __init__(self, content=b"", status=204): + self.content = io.BytesIO(content) + self.status = status + self.headers = Message() + self.headers["Content-Type"] = "text/event-stream" + + def __enter__(self): + return self + + def __exit__(self, *args): + return False + + def readline(self, limit): + return self.content.readline(limit) + + +class TeamMemoryTests(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls.client = action.load_client() + + def setUp(self): + self.temporary = tempfile.TemporaryDirectory() + self.addCleanup(self.temporary.cleanup) + self.directory = Path(self.temporary.name) + self.repository = "microsoft/vscode-gradle" + self.before, self.merge, self.after, self.tip = [letter * 40 for letter in "daef"] + self.project = self.make_project(self.repository) + self.central = self.make_project(action.COORDINATOR) + self.environment = { + "GITHUB_REPOSITORY": action.COORDINATOR, "GITHUB_EVENT_NAME": "workflow_dispatch", + "GITHUB_REF": "refs/heads/main", "GITHUB_SHA": self.tip, "GITHUB_WORKFLOW_SHA": self.tip, + "GITHUB_WORKFLOW_REF": action.COORDINATOR + "/" + action.COORDINATOR_WORKFLOW + "@refs/heads/main", + "GITHUB_ACTOR": "maintainer", "GITHUB_TRIGGERING_ACTOR": "rerunner", + "GITHUB_RUN_ID": "999", "GITHUB_RUN_ATTEMPT": "1", + "GITHUB_EVENT_PATH": str(self.directory / "event.json"), "RUNNER_TEMP": str(self.directory), + "GITHUB_OUTPUT": str(self.directory / "output.txt"), + "COORDINATOR_TOKEN": "test-coordinator-token", "GH_TOKEN": "test-source-token", + "DISPATCH_TOKEN": "test-dispatch-token", "SOURCE_REPOSITORY": self.repository, + "SOURCE_RUN_ID": "123456", "SOURCE_RUN_ATTEMPT": "2", "SOURCE_ARTIFACT_ID": "456", + "DISPATCH_PR": "", "OUTPUT_MODE": "quiet", "SUMMARY_MODE": "none", + "AGENT_URL": "https://test.services.ai.azure.com/agents/test/protocols/invocations", + "AGENT_SCOPE": "api://test/.default", + } + self.env_patch = patch.dict(os.environ, self.environment, clear=True) + self.env_patch.start() + self.addCleanup(self.env_patch.stop) + self.write_event({"repository": self.central}) + self.source_run = { + "id": 123456, "run_attempt": 2, "event": "push", "path": action.SOURCE_WORKFLOW, + "head_branch": "develop", "head_sha": self.after, + "repository": self.project, "head_repository": self.project, + "actor": {"login": "maintainer"}, "triggering_actor": {"login": "rerunner"}, + } + now = datetime.now(timezone.utc) - timedelta(minutes=1) + self.artifact = { + "id": 456, "name": "issuelens-team-memory-source-2", "expired": False, "size_in_bytes": 2048, + "digest": "sha256:" + "1" * 64, "created_at": now.isoformat(), + "expires_at": (now + timedelta(days=7)).isoformat(), + "workflow_run": { + "id": 123456, "repository_id": self.project["id"], "head_repository_id": self.project["id"], + "head_branch": "develop", "head_sha": self.after, + }, + } + self.metadata = { + "repository": self.repository, "repository_id": self.project["id"], "base_ref": "develop", + "event_name": "push", "event_action": "push", "actor_login": "maintainer", "triggering_actor": "rerunner", + "workflow_ref": self.repository + "/" + action.SOURCE_WORKFLOW + "@refs/heads/develop", + "workflow_sha": self.after, "run_id": 123456, "run_attempt": 2, + } + self.push = { + "repository": {"id": self.project["id"], "full_name": self.repository}, "ref": "refs/heads/develop", + "before": self.before, "after": self.after, "created": False, "deleted": False, "forced": False, + "commits": [{"id": self.merge}, {"id": self.after}], "head_commit": {"id": self.after}, + } + self.snapshot = {"metadata": self.metadata, "event": self.push} + self.pull = { + "number": 27, "merged": True, "state": "closed", "merge_commit_sha": self.merge, + "merged_at": "2026-10-08T00:00:00Z", "base": {"ref": "develop", "repo": self.project}, + } + self.comparison = { + "base_commit": {"sha": self.before}, "merge_base_commit": {"sha": self.before}, "status": "ahead", + "ahead_by": 2, "behind_by": 0, "total_commits": 2, "commits": [{"sha": self.after}], + } + node = { + "number": 27, "state": "MERGED", "merged": True, "mergedAt": self.pull["merged_at"], + "baseRefName": "develop", "baseRepository": {"databaseId": self.project["id"], "nameWithOwner": self.repository}, + "mergeCommit": {"oid": self.merge}, + } + connection = {"totalCount": 1, "pageInfo": {"hasNextPage": False}, "nodes": [node]} + self.graphql = {"data": {"repository": { + "databaseId": self.project["id"], "nameWithOwner": self.repository, + "defaultBranchRef": {"name": "develop", "target": {"oid": self.tip}}, + "c0": {"oid": self.merge, "associatedPullRequests": copy.deepcopy(connection)}, + "c1": {"oid": self.after, "associatedPullRequests": copy.deepcopy(connection)}, + }}} + self.api_reads = [] + self.mock_read = patch.object(action, "github_read", side_effect=self.api_read).start() + self.addCleanup(patch.stopall) + patch.object(self.client, "github_read", side_effect=lambda path, payload=None: self.api_read( + path, os.environ["GH_TOKEN"], payload)).start() + patch.object(action, "load_client", return_value=self.client).start() + self.opener = Mock() + self.opener.open.return_value = Response() + patch("urllib.request.build_opener", return_value=self.opener).start() + self.azure = patch.object(self.client.subprocess, "check_output", return_value="test-azure-token").start() + self.log = patch("sys.stdout", new_callable=io.StringIO).start() + + def make_project(self, repository): + identity, branch = action.SOURCES[repository] + return {"full_name": repository, "id": identity, "default_branch": branch, "private": False, "archived": False} + + def api_read(self, path, token, payload=None): + self.api_reads.append((path, token, payload)) + responses = { + f"/repos/{action.COORDINATOR}": self.central, f"/repos/{self.repository}": self.project, + f"/repos/{self.repository}/actions/runs/123456/attempts/2": self.source_run, + f"/repos/{self.repository}/actions/artifacts/456": self.artifact, + f"/repos/{self.repository}/pulls/27": self.pull, + f"/repos/{self.repository}/compare/{self.before}...{self.after}?per_page=1&page=2": self.comparison, + "/graphql": self.graphql, + } + self.assertIn(path, responses, "Unexpected API access") + return copy.deepcopy(responses[path]) + + def write_event(self, event): + Path(os.environ["GITHUB_EVENT_PATH"]).write_text(json.dumps(event), encoding="utf-8") + + def write_source(self, content=None): + path = action.source_event_path() + path.parent.mkdir(exist_ok=True) + path.write_bytes(json.dumps(self.snapshot).encode() if content is None else content) + return path + + def output(self): + path = Path(os.environ["GITHUB_OUTPUT"]) + return dict(line.split("=", 1) for line in path.read_text().splitlines()) if path.exists() else {} + + def envelope(self): + return json.loads(Path(self.output()["request-path"]).read_text()) + + def select_push(self): + os.environ.update( + GITHUB_REPOSITORY=self.repository, GITHUB_EVENT_NAME="push", GITHUB_SHA=self.after, + GITHUB_WORKFLOW_SHA=self.after, GITHUB_REF="refs/heads/develop", + GITHUB_WORKFLOW_REF=self.metadata["workflow_ref"], GITHUB_RUN_ID="123456", GITHUB_RUN_ATTEMPT="2", + ) + self.write_event(self.push) + + def execute(self, command): + action.run(command) + self.azure.assert_not_called() + + def test_identity_only_source_excludes_all_untrusted_text_and_tokens(self): + self.select_push() + event = copy.deepcopy(self.push) + event["commits"][0]["message"] = "UNTRUSTED_COMMIT_MESSAGE" + event["repository"]["description"] = "UNTRUSTED_REPOSITORY_TEXT" + self.write_event(event) + self.execute("prepare-dispatch") + path = Path(self.output()["source-event-path"]) + self.assertEqual(json.loads(path.read_bytes()), self.snapshot) + self.assertNotIn("UNTRUSTED", path.read_text()) + self.assertNotIn("test-source-token", path.read_text()) + self.opener.open.assert_not_called() + + def test_maximum_inventory_fits_and_matches_upstream_limits(self): + self.assertEqual(action.MAX_PUSH_COMMITS, self.client.MAX_PUSH_COMMITS) + self.assertEqual(action.MAX_SOURCE_BYTES, self.client.MAX_SOURCE_BYTES) + self.select_push() + event = copy.deepcopy(self.push) + event["commits"] = [{"id": f"{number:040x}"} for number in range(1, 1000)] + [{"id": self.after}] + self.write_event(event) + self.execute("prepare-dispatch") + self.assertLess(action.source_event_path().stat().st_size, action.MAX_SOURCE_BYTES) + + def test_every_allowlisted_source_uses_its_own_identity_and_default_branch(self): + original = dict(os.environ) + for repository, (identity, branch) in action.SOURCES.items(): + with self.subTest(repository=repository), patch.dict(os.environ, { + **original, "GITHUB_REPOSITORY": repository, "GITHUB_EVENT_NAME": "push", + "GITHUB_REF": "refs/heads/" + branch, "GITHUB_SHA": self.after, "GITHUB_WORKFLOW_SHA": self.after, + "GITHUB_WORKFLOW_REF": repository + "/" + action.SOURCE_WORKFLOW + "@refs/heads/" + branch, + }, clear=True), patch.object(action, "github_read", return_value=self.make_project(repository)): + self.write_event({**self.push, "repository": {"id": identity, "full_name": repository}, + "ref": "refs/heads/" + branch}) + snapshot = action.push_snapshot() + self.assertEqual(snapshot["metadata"]["repository_id"], identity) + self.assertEqual(snapshot["metadata"]["base_ref"], branch) + self.assertEqual(snapshot["event"]["repository"]["full_name"], repository) + + def test_push_rejects_unsafe_or_incomplete_identities(self): + self.select_push() + cases = [ + {"forced": True}, {"created": True}, {"deleted": True}, {"before": "0" * 40}, + {"after": self.before}, {"head_commit": {"id": self.merge}}, {"ref": "refs/heads/main"}, + {"commits": []}, {"commits": [{"id": self.after}] * 2}, + {"commits": [{"id": self.after}] * 1001}, {"commits": [{"id": "short"}]}, + ] + for changes in cases: + with self.subTest(changes=changes): + self.write_event({**self.push, **changes}) + with self.assertRaises(SystemExit): + self.execute("prepare-dispatch") + self.assertFalse(action.source_event_path().exists()) + self.assertEqual(self.output(), {}) + + def test_push_rejects_wrong_workflow_branch_revision_and_repository(self): + self.select_push() + original = dict(os.environ) + for name, value in ( + ("GITHUB_WORKFLOW_REF", self.repository + "/.github/workflows/untrusted.yml@refs/heads/develop"), + ("GITHUB_REF", "refs/heads/main"), ("GITHUB_WORKFLOW_SHA", self.tip), + ("GITHUB_EVENT_NAME", "pull_request"), ("GITHUB_REPOSITORY", "microsoft/vscode-spring-initializr"), + ): + with self.subTest(name=name), patch.dict(os.environ, {**original, name: value}, clear=True): + with self.assertRaises(SystemExit): + self.execute("prepare-dispatch") + self.assertEqual(self.output(), {}) + + def test_dispatch_sends_one_post_to_central_main_with_separate_tokens(self): + self.select_push() + self.write_source() + self.execute("dispatch") + self.assertEqual([(path, token) for path, token, _ in self.api_reads], [ + (f"/repos/{self.repository}", "test-source-token"), + (f"/repos/{action.COORDINATOR}", "test-dispatch-token"), + ]) + self.opener.open.assert_called_once() + request = self.opener.open.call_args.args[0] + self.assertEqual(request.full_url, + f"https://api.github.com/repos/{action.COORDINATOR}/actions/workflows/team-memory-coordinator.yml/dispatches") + self.assertEqual(request.get_method(), "POST") + self.assertEqual(json.loads(request.data), {"ref": "main", "inputs": { + "source_repository": self.repository, "source_run_id": "123456", + "source_run_attempt": "2", "source_artifact_id": "456", + }}) + self.assertNotIn(b"test-", request.data) + self.assertIn("completion is reported by the coordinator", self.log.getvalue()) + + def test_dispatch_does_not_retry_ambiguous_failure_or_leak_transport_details(self): + self.select_push() + self.write_source() + self.opener.open.side_effect = OSError("PRIVATE_TRANSPORT_DETAIL") + with self.assertRaisesRegex(SystemExit, "outcome is unknown"): + self.execute("dispatch") + self.opener.open.assert_called_once() + self.assertNotIn("PRIVATE_TRANSPORT_DETAIL", self.log.getvalue()) + self.assertEqual(self.output(), {}) + + def test_explicit_empty_dispatch_token_fails_before_any_network_access(self): + self.select_push() + for token in ("", " \t"): + with self.subTest(token=token), patch.dict(os.environ, {"DISPATCH_TOKEN": token}): + with self.assertRaisesRegex(SystemExit, "dispatch-token must be non-empty"): + self.execute("dispatch") + self.mock_read.assert_not_called() + self.opener.open.assert_not_called() + + def test_dispatch_revalidates_the_original_event_not_only_saved_metadata(self): + self.select_push() + self.snapshot["event"]["before"] = self.merge + self.write_source() + with self.assertRaisesRegex(SystemExit, "Source event changed"): + self.execute("dispatch") + self.opener.open.assert_not_called() + + def test_invalid_mixed_and_missing_inputs_fail_before_network(self): + cases = [ + {"SOURCE_REPOSITORY": ""}, {"SOURCE_REPOSITORY": "microsoft/IssueLens"}, + {"SOURCE_RUN_ID": ""}, {"SOURCE_RUN_ATTEMPT": ""}, {"SOURCE_ARTIFACT_ID": ""}, + {"SOURCE_RUN_ID": "1; unsafe"}, {"SOURCE_RUN_ATTEMPT": "0"}, {"SOURCE_ARTIFACT_ID": "01"}, + {"DISPATCH_PR": "27"}, {name: "" for name in action.SOURCE_INPUTS}, + ] + for changes in cases: + with self.subTest(changes=changes), patch.dict(os.environ, changes): + with self.assertRaises(SystemExit): + self.execute("select-source") + self.mock_read.assert_not_called() + + def test_only_central_default_branch_coordinator_may_select_a_source(self): + for change in ( + {"GITHUB_REPOSITORY": self.repository}, {"GITHUB_EVENT_NAME": "push"}, + {"GITHUB_REF": "refs/heads/feature"}, {"GITHUB_WORKFLOW_SHA": "short"}, + {"GITHUB_WORKFLOW_REF": action.COORDINATOR + "/.github/workflows/untrusted.yml@refs/heads/main"}, + ): + with self.subTest(change=change), patch.dict(os.environ, change): + with self.assertRaises(SystemExit): + self.execute("select-source") + self.assertEqual(self.output(), {}) + + def test_source_selection_precedes_external_token_minting(self): + self.execute("select-source") + self.assertEqual(self.output(), {"external": "true", "source-name": "vscode-gradle"}) + self.assertEqual(self.api_reads, [(f"/repos/{action.COORDINATOR}", "test-coordinator-token", None)]) + + def test_java_pack_manual_source_needs_no_external_credentials(self): + self.repository = action.COORDINATOR + self.project = self.central + self.pull["base"] = {"ref": "main", "repo": self.central} + os.environ.update({**{name: "" for name in action.SOURCE_INPUTS}, + "DISPATCH_PR": "27", "SOURCE_REPOSITORY": action.COORDINATOR, + "GH_TOKEN": "test-coordinator-token"}) + self.execute("select-source") + self.assertEqual(self.output(), {"external": "false", "source-name": "vscode-java-pack"}) + Path(os.environ["GITHUB_OUTPUT"]).unlink() + self.execute("preflight") + metadata = self.envelope()["metadata"] + self.assertEqual(metadata["repository"], action.COORDINATOR) + self.assertEqual(metadata["base_ref"], "main") + self.assertEqual(metadata["required_wiki_repository"], action.COORDINATOR) + self.assertTrue(all(token == "test-coordinator-token" for _, token, _ in self.api_reads)) + + def test_validates_external_source_run_and_artifact_before_download(self): + self.execute("validate-dispatch") + self.assertEqual(self.output(), { + "automatic": "true", "source-repository": self.repository, + "source-run-id": "123456", "source-artifact-id": "456", + }) + self.assertEqual([path for path, _, _ in self.api_reads], [ + f"/repos/{action.COORDINATOR}", f"/repos/{self.repository}", + f"/repos/{self.repository}/actions/runs/123456/attempts/2", + f"/repos/{self.repository}/actions/artifacts/456", + ]) + self.assertTrue(all(token == "test-source-token" for _, token, _ in self.api_reads[1:])) + self.opener.open.assert_not_called() + + def test_rejects_changed_identity_visibility_or_default_branch(self): + original = copy.deepcopy(self.project) + for changes in ( + {"id": original["id"] + 1}, {"full_name": "microsoft/other"}, {"default_branch": "main"}, + {"private": True}, {"private": None}, {"archived": True}, + ): + with self.subTest(changes=changes): + self.project = {**original, **changes} + with self.assertRaises(SystemExit): + self.execute("validate-dispatch") + self.assertFalse(any("/actions/" in path for path, _, _ in self.api_reads)) + + def test_forged_run_cannot_proceed_to_artifact_validation(self): + original = copy.deepcopy(self.source_run) + for changes in ( + {"id": 123457}, {"run_attempt": 1}, {"event": "workflow_dispatch"}, {"path": "untrusted.yml"}, + {"head_branch": "main"}, {"head_sha": "short"}, + {"repository": self.central}, {"head_repository": self.central}, + {"actor": {"login": "unsafe\nactor"}}, + ): + with self.subTest(changes=changes): + self.source_run = {**original, **changes} + with self.assertRaises(SystemExit): + self.execute("validate-dispatch") + self.assertFalse(any("/artifacts/" in path for path, _, _ in self.api_reads)) + + def test_foreign_expired_oversized_or_digestless_artifacts_fail(self): + original = copy.deepcopy(self.artifact) + for changes in ( + {"id": 457}, {"name": "issuelens-team-memory-source-1"}, {"expired": True}, {"digest": None}, + {"digest": "short"}, {"size_in_bytes": 0}, {"size_in_bytes": True}, + {"size_in_bytes": action.MAX_SOURCE_BYTES + 1}, + {"workflow_run": {**original["workflow_run"], "id": 123457}}, + {"workflow_run": {**original["workflow_run"], "repository_id": self.central["id"]}}, + {"workflow_run": {**original["workflow_run"], "head_sha": self.merge}}, + ): + with self.subTest(changes=changes): + self.artifact = {**original, **changes} + with self.assertRaises(SystemExit): + self.execute("validate-dispatch") + self.assertEqual(self.output(), {}) + + def test_expiry_timestamps_are_checked_not_only_expired_flag(self): + original = copy.deepcopy(self.artifact) + now = datetime.now(timezone.utc) + for changes in ( + {"expires_at": (now - timedelta(seconds=1)).isoformat()}, + {"expires_at": (now + timedelta(days=8)).isoformat()}, + {"created_at": (now + timedelta(hours=1)).isoformat()}, + {"expires_at": "2026-10-01T00:00:00"}, {"created_at": "invalid"}, + ): + with self.subTest(changes=changes): + self.artifact = {**original, **changes} + with self.assertRaises(SystemExit): + self.execute("validate-dispatch") + + def test_preflight_reuses_discovery_and_keeps_source_and_coordinator_separate(self): + self.write_source() + self.execute("preflight") + envelope = self.envelope() + metadata = envelope["metadata"] + self.assertEqual(metadata["repository"], self.repository) + self.assertEqual(metadata["base_ref"], "develop") + self.assertEqual(metadata["workflow_sha"], self.after) + self.assertEqual(metadata["coordinator_repository"], action.COORDINATOR) + self.assertEqual(metadata["coordinator_workflow_sha"], self.tip) + self.assertEqual(metadata["required_wiki_repository"], action.COORDINATOR) + self.assertEqual(metadata["run_id"], 123456) + self.assertEqual(metadata["coordinator_run_id"], 999) + self.assertEqual(metadata["source_tip_sha"], self.tip) + self.assertEqual([item["pull_number"] for item in metadata["pull_requests"]], [27]) + self.assertEqual(envelope["request_type"], "team-memory") + self.assertIn("This never overrides repository policy", envelope["request"]["input"]) + self.assertIn("Do not modify source or issues", envelope["request"]["input"]) + self.assertIn("current wiki knowledge", envelope["request"]["input"]) + self.assertTrue(any("/actions/artifacts/" in path for path, _, _ in self.api_reads)) + + def test_malformed_or_forged_downloaded_snapshot_is_rejected(self): + original = copy.deepcopy(self.snapshot) + cases = [ + {**original, "secret": "forbidden"}, + {**original, "metadata": {**original["metadata"], "run_id": 999}}, + {**original, "metadata": {**original["metadata"], "run_attempt": 2.0}}, + {**original, "metadata": {**original["metadata"], "repository": action.COORDINATOR}}, + {**original, "event": {**original["event"], "message": "forbidden"}}, + {**original, "event": {**original["event"], "commits": [{"id": self.after, "message": "forbidden"}]}}, + {**original, "event": {**original["event"], "repository": {"id": self.central["id"], "full_name": action.COORDINATOR}}}, + {**original, "event": {**original["event"], "after": self.merge}}, + ] + for snapshot in cases: + with self.subTest(snapshot=snapshot): + self.write_source(json.dumps(snapshot).encode()) + with self.assertRaises(SystemExit): + self.execute("preflight") + for content in (b"{}", b'{"metadata":{},"metadata":{},"event":{}}', b"x" * (action.MAX_SOURCE_BYTES + 1)): + with self.subTest(content=content[:50]): + self.write_source(content) + with self.assertRaises(SystemExit): + self.execute("preflight") + self.assertEqual(self.output(), {}) + + def test_discovery_rejects_incomplete_range_and_association_pagination(self): + self.write_source() + self.comparison["total_commits"] = 3 + with self.assertRaisesRegex(SystemExit, "complete fast-forward"): + self.execute("preflight") + self.comparison["total_commits"] = 2 + self.graphql["data"]["repository"]["c0"]["associatedPullRequests"]["pageInfo"]["hasNextPage"] = True + with self.assertRaisesRegex(SystemExit, "associations are incomplete"): + self.execute("preflight") + self.assertEqual(self.output(), {}) + + def test_rebased_merge_uses_pinned_rest_revalidation(self): + self.write_source() + for key in ("c0", "c1"): + self.graphql["data"]["repository"][key]["associatedPullRequests"]["nodes"][0]["mergeCommit"] = None + self.execute("preflight") + self.assertEqual(self.envelope()["metadata"]["pull_requests"][0]["merge_commit_sha"], self.merge) + self.assertEqual(sum(path.endswith("/pulls/27") for path, _, _ in self.api_reads), 1) + + def test_no_merged_pr_is_an_explicit_skip_before_login(self): + self.write_source() + for key in ("c0", "c1"): + self.graphql["data"]["repository"][key]["associatedPullRequests"] = { + "totalCount": 0, "pageInfo": {"hasNextPage": False}, "nodes": [], + } + self.execute("preflight") + self.assertEqual(self.output(), {"eligible": "false", "status": "skipped", "skip-reason": "no_merged_pull_requests"}) + + def test_manual_external_pr_uses_source_default_branch_and_same_wiki(self): + os.environ.update({**{name: "" for name in action.SOURCE_INPUTS}, "DISPATCH_PR": "27"}) + self.execute("validate-dispatch") + self.assertEqual(self.output(), {"automatic": "false", "source-repository": self.repository}) + Path(os.environ["GITHUB_OUTPUT"]).unlink() + self.execute("preflight") + metadata = self.envelope()["metadata"] + self.assertEqual(metadata["repository"], self.repository) + self.assertEqual(metadata["base_ref"], "develop") + self.assertEqual(metadata["pull_number"], 27) + self.assertEqual(metadata["required_wiki_repository"], action.COORDINATOR) + self.assertNotIn("workflow_sha", metadata) + self.assertFalse(any("/actions/" in path for path, _, _ in self.api_reads)) + + def test_manual_unmerged_or_wrong_source_pr_is_rejected(self): + os.environ.update({**{name: "" for name in action.SOURCE_INPUTS}, "DISPATCH_PR": "27"}) + original = copy.deepcopy(self.pull) + for changes in ( + {"merged": False}, {"state": "open"}, {"merge_commit_sha": "short"}, + {"base": {"ref": "main", "repo": self.project}}, + {"base": {"ref": "develop", "repo": self.central}}, + ): + with self.subTest(changes=changes): + self.pull = {**original, **changes} + with self.assertRaises(SystemExit): + self.execute("preflight") + self.assertEqual(self.output(), {}) + + def result(self, status="no-change"): + return { + "source_repository": self.repository, "push_before": self.before, "push_after": self.after, + "status": status, "wiki_repository": action.COORDINATOR, "wiki_sha": "b" * 40, + "reason": "Verified source and wiki", "results": [ + {"pull_number": 27, "merge_commit_sha": self.merge, "status": status, "reason": "Verified"}, + ], + } + + def submit(self, result, done=True): + os.environ["REQUEST_PATH"] = self.output()["request-path"] + stream = 'data: ' + json.dumps({"type": "assistant.message", "data": {"content": json.dumps(result)}}) + "\n\n" + if done: + stream += 'event: done\ndata: {"invocation_id":"test","session_id":"test"}\n\n' + self.opener.open.return_value = Response(stream.encode()) + self.client.run("submit") + + def test_completed_stream_with_valid_receipt_is_required_for_success(self): + self.write_source() + self.execute("preflight") + self.submit(self.result()) + self.assertEqual(self.output()["status"], "no-change") + self.assertEqual(self.output()["wiki-repository"], action.COORDINATOR) + self.opener.open.assert_called_once() + + def test_stream_completion_alone_wrong_wiki_missing_pr_or_receipt_is_not_success(self): + self.write_source() + self.execute("preflight") + original = self.result() + for changes in ( + {"wiki_repository": "microsoft/IssueLens"}, {"wiki_sha": None}, {"results": []}, + {"source_repository": action.COORDINATOR}, {"push_after": self.merge}, {"status": "completed"}, + ): + with self.subTest(changes=changes): + with self.assertRaises(SystemExit): + self.submit({**original, **changes}) + self.assertNotIn("status", self.output()) + + def test_incomplete_stream_fails_without_retry(self): + self.write_source() + self.execute("preflight") + with self.assertRaisesRegex(SystemExit, "completion event"): + self.submit(self.result(), done=False) + self.opener.open.assert_called_once() + self.assertNotIn("status", self.output()) + + def test_failed_or_partial_batch_does_not_claim_whole_batch_success(self): + self.write_source() + self.execute("preflight") + with self.assertRaisesRegex(SystemExit, "batch incomplete"): + self.submit(self.result("failed")) + self.assertEqual(self.output()["status"], "failed") + self.opener.open.assert_called_once() + + def test_partial_publication_is_reported_but_fails_the_coordinator(self): + self.write_source() + connection = self.graphql["data"]["repository"]["c1"]["associatedPullRequests"] + connection["nodes"].append({**copy.deepcopy(connection["nodes"][0]), + "number": 28, "mergeCommit": {"oid": self.after}}) + connection["totalCount"] = 2 + self.execute("preflight") + result = self.result() + result["status"] = "partial" + result["results"].append({"pull_number": 28, "merge_commit_sha": self.after, + "status": "needs-review", "reason": "Incomplete evidence"}) + with self.assertRaisesRegex(SystemExit, "batch incomplete"): + self.submit(result) + self.assertEqual(self.output()["status"], "partial") + self.assertEqual(self.output()["wiki-repository"], action.COORDINATOR) + self.opener.open.assert_called_once() + + +class WiringTests(unittest.TestCase): + def test_only_verified_sources_are_allowlisted(self): + self.assertEqual(len(action.SOURCES), 8) + self.assertNotIn("microsoft/vscode-spring-initializr", action.SOURCES) + self.assertNotIn("microsoft/vscode-spring-boot-dashboard", action.SOURCES) + self.assertEqual(action.SOURCES["microsoft/build-server-for-gradle"][1], "develop") + + def test_workflows_share_new_opt_in_and_only_coordinator_invokes(self): + source = (ROOT / ".github/workflows/team-memory-post-merge.yml").read_text() + central = (ROOT / ".github/workflows/team-memory-coordinator.yml").read_text() + composite = (ACTION / "action.yml").read_text() + for workflow in (source, central): + self.assertIn("ISSUELENS_TEAM_MEMORY_COORDINATOR_ENABLED == 'true'", workflow) + self.assertNotIn("ISSUELENS_TEAM_MEMORY_ENABLED", workflow) + self.assertIn("permissions: {}", workflow) + self.assertIn("persist-credentials: false", workflow) + self.assertNotIn("concurrency:", source) + self.assertNotIn("workflow_dispatch:", source) + for text in (source, composite): + self.assertNotIn("azure/login", text) + self.assertNotIn("agent-url", text) + self.assertNotIn("id-token:", text) + self.assertIn("group: issuelens-team-memory-wiki-microsoft-vscode-java-pack", central) + self.assertIn("queue: max\n cancel-in-progress: false", central) + self.assertIn("digest-mismatch: error", central) + self.assertNotIn("steps.source-token.outputs.token || github.token", central) + self.assertIn("repository: ${{ steps.source.outputs.source-repository }}", central) + self.assertLess(central.index("validate-dispatch"), central.index("actions/download-artifact@")) + self.assertLess(central.index("team_memory.py preflight"), central.index("azure/login@")) + self.assertIn('python3 -I "$ISSUELENS_CLIENT_PATH" submit', central) + self.assertIn("retention-days: 7", composite) + self.assertIn("GH_TOKEN: ${{ inputs.source-token }}", composite) + self.assertIn("DISPATCH_TOKEN: ${{ inputs.dispatch-token }}", composite) + + def test_client_pin_matches_coordinator_tests_and_documentation(self): + revision = "4175ea71e170938826fb847e6bd5108f0f5597cf" + for relative in (".github/workflows/team-memory-coordinator.yml", ".github/workflows/team-memory-tests.yml"): + self.assertIn("ref: " + revision, (ROOT / relative).read_text()) + self.assertIn(revision, (ACTION / "README.md").read_text()) + + +if __name__ == "__main__": + unittest.main() diff --git a/.github/workflows/team-memory-coordinator.yml b/.github/workflows/team-memory-coordinator.yml new file mode 100644 index 00000000..cf444740 --- /dev/null +++ b/.github/workflows/team-memory-coordinator.yml @@ -0,0 +1,137 @@ +name: Coordinate Java team memory (opt-in) + +on: + workflow_dispatch: + inputs: + source_repository: + description: Allowlisted Java source repository (owner/name) + required: true + default: microsoft/vscode-java-pack + type: string + source_run_id: + description: Source push workflow run ID (automatic dispatch only) + required: false + type: string + source_run_attempt: + description: Source workflow attempt (required with source_run_id) + required: false + type: string + source_artifact_id: + description: Immutable source event artifact ID (required with source_run_id) + required: false + type: string + pull_request_number: + description: Merged source PR to reconcile manually (instead of source inputs) + required: false + type: string + +permissions: {} + +concurrency: + group: issuelens-team-memory-wiki-microsoft-vscode-java-pack + queue: max + cancel-in-progress: false + +jobs: + reconcile: + if: >- + vars.ISSUELENS_TEAM_MEMORY_COORDINATOR_ENABLED == 'true' && + github.repository == 'microsoft/vscode-java-pack' && + github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && + github.event_name == 'workflow_dispatch' + runs-on: ubuntu-latest + timeout-minutes: 30 + permissions: + contents: read + pull-requests: read + actions: read + id-token: write + env: + COORDINATOR_TOKEN: ${{ github.token }} + SOURCE_REPOSITORY: ${{ inputs.source_repository }} + SOURCE_RUN_ID: ${{ inputs.source_run_id }} + SOURCE_RUN_ATTEMPT: ${{ inputs.source_run_attempt }} + SOURCE_ARTIFACT_ID: ${{ inputs.source_artifact_id }} + DISPATCH_PR: ${{ inputs.pull_request_number }} + ISSUELENS_CLIENT_PATH: .issuelens-client/.github/actions/issuelens/issuelens_action.py + OUTPUT_MODE: activity + SUMMARY_MODE: status + steps: + - name: Load Java adapter from trusted workflow revision + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ github.workflow_sha }} + persist-credentials: false + sparse-checkout: /.github/actions/queue-team-memory/ + sparse-checkout-cone-mode: false + + - name: Load reviewed IssueLens client + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + repository: microsoft/IssueLens + ref: 4175ea71e170938826fb847e6bd5108f0f5597cf # merged PR 50 + path: .issuelens-client + persist-credentials: false + sparse-checkout: /.github/actions/issuelens/ + sparse-checkout-cone-mode: false + + - name: Validate coordinator and source selection + id: selection + shell: bash + run: python3 -I .github/actions/queue-team-memory/team_memory.py select-source + + - name: Get read-only token for the selected external source + if: steps.selection.outputs.external == 'true' + id: source-token + uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3 + with: + client-id: ${{ secrets.ISSUELENS_SOURCE_READ_APP_CLIENT_ID }} + private-key: ${{ secrets.ISSUELENS_SOURCE_READ_APP_PRIVATE_KEY }} + owner: microsoft + repositories: ${{ steps.selection.outputs.source-name }} + permission-actions: read + permission-contents: read + permission-pull-requests: read + + - name: Verify queued source before download + id: source + shell: bash + env: + GH_TOKEN: ${{ steps.selection.outputs.external == 'false' && github.token || steps.source-token.outputs.token }} + run: python3 -I .github/actions/queue-team-memory/team_memory.py validate-dispatch + + - name: Download verified source event + if: steps.source.outputs.automatic == 'true' + uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + artifact-ids: ${{ steps.source.outputs.source-artifact-id }} + run-id: ${{ steps.source.outputs.source-run-id }} + repository: ${{ steps.source.outputs.source-repository }} + github-token: ${{ steps.selection.outputs.external == 'false' && github.token || steps.source-token.outputs.token }} + path: ${{ runner.temp }}/issuelens-team-memory-source + digest-mismatch: error + + - name: Revalidate source and prepare bounded maintenance request + id: preflight + shell: bash + env: + GH_TOKEN: ${{ steps.selection.outputs.external == 'false' && github.token || steps.source-token.outputs.token }} + run: python3 -I .github/actions/queue-team-memory/team_memory.py preflight + + - name: Azure login for the existing agent endpoint + if: steps.preflight.outputs.eligible == 'true' + uses: azure/login@7ddb5af1ef8758cf1353cf3b42f940aee27ba21c # v3.0.2 + with: + client-id: ${{ secrets.AZURE_CLIENT_ID }} + tenant-id: ${{ secrets.AZURE_TENANT_ID }} + subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }} + + - name: Invoke IssueLens and validate the final maintenance receipt + if: steps.preflight.outputs.eligible == 'true' + shell: bash + env: + GH_TOKEN: ${{ steps.selection.outputs.external == 'false' && github.token || steps.source-token.outputs.token }} + REQUEST_PATH: ${{ steps.preflight.outputs.request-path }} + AGENT_URL: ${{ secrets.ISSUELENS_AGENT_URL }} + AGENT_SCOPE: ${{ secrets.ISSUELENS_AGENT_SCOPE }} + run: python3 -I "$ISSUELENS_CLIENT_PATH" submit diff --git a/.github/workflows/team-memory-post-merge.yml b/.github/workflows/team-memory-post-merge.yml index 0bc11414..e9295e71 100644 --- a/.github/workflows/team-memory-post-merge.yml +++ b/.github/workflows/team-memory-post-merge.yml @@ -1,43 +1,31 @@ -name: Update IssueLens Team Memory +name: Queue Java team memory (opt-in) on: push: branches: [main] - workflow_dispatch: - inputs: - pull_request_number: - description: 'Merged pull request number to learn from' - required: true - type: string permissions: {} -concurrency: - group: issuelens-${{ github.repository }}-team-memory-${{ github.event.after || inputs.pull_request_number || github.run_id }} - cancel-in-progress: false - jobs: - team-memory: + dispatch: if: >- - ${{ vars.ISSUELENS_TEAM_MEMORY_ENABLED == 'true' && + ${{ vars.ISSUELENS_TEAM_MEMORY_COORDINATOR_ENABLED == 'true' && + github.repository == 'microsoft/vscode-java-pack' && github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && - (github.event_name == 'push' || github.event_name == 'workflow_dispatch') }} + github.event_name == 'push' }} runs-on: ubuntu-latest - timeout-minutes: 30 + timeout-minutes: 5 permissions: contents: read - pull-requests: read - id-token: write + actions: write steps: - - name: Maintain IssueLens team memory - uses: microsoft/IssueLens/.github/actions/issuelens@2b5317815e15179899014a02c41678620ce2d390 + - name: Load dispatcher from trusted push revision + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: - request-type: team-memory - pull-request-number: ${{ inputs.pull_request_number }} - azure-client-id: ${{ secrets.AZURE_CLIENT_ID }} - azure-tenant-id: ${{ secrets.AZURE_TENANT_ID }} - azure-subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }} - agent-url: ${{ secrets.ISSUELENS_AGENT_URL }} - agent-scope: ${{ secrets.ISSUELENS_AGENT_SCOPE }} - output-mode: activity - summary-mode: status + ref: ${{ github.workflow_sha }} + persist-credentials: false + sparse-checkout: /.github/actions/queue-team-memory/ + sparse-checkout-cone-mode: false + + - name: Queue team-memory update + uses: ./.github/actions/queue-team-memory diff --git a/.github/workflows/team-memory-tests.yml b/.github/workflows/team-memory-tests.yml new file mode 100644 index 00000000..cb9e2c85 --- /dev/null +++ b/.github/workflows/team-memory-tests.yml @@ -0,0 +1,39 @@ +name: Java team-memory adapter checks + +on: + pull_request: + paths: + - .github/actions/queue-team-memory/** + - .github/workflows/team-memory*.yml + push: + branches: [main] + paths: + - .github/actions/queue-team-memory/** + - .github/workflows/team-memory*.yml + +permissions: {} + +jobs: + test: + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + + - name: Load reviewed IssueLens client + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + repository: microsoft/IssueLens + ref: 4175ea71e170938826fb847e6bd5108f0f5597cf + path: .issuelens-client + persist-credentials: false + sparse-checkout: /.github/actions/issuelens/ + sparse-checkout-cone-mode: false + + - name: Check source trust, dispatch, discovery, and receipts offline + env: + ISSUELENS_CLIENT_PATH: .issuelens-client/.github/actions/issuelens/issuelens_action.py + run: python3 -m unittest discover -s .github/actions/queue-team-memory/tests From 9e71e5eab5845ffcf85c92c52171868b60367e07 Mon Sep 17 00:00:00 2001 From: Changyong Gong Date: Fri, 9 Oct 2026 14:50:35 +0800 Subject: [PATCH 2/5] Reuse pinned IssueLens team-memory dispatcher Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- .github/actions/queue-team-memory/README.md | 43 ++++++--- .github/actions/queue-team-memory/action.yml | 46 --------- .../actions/queue-team-memory/team_memory.py | 95 ++----------------- .../tests/test_team_memory.py | 88 ++++++++++++++--- .github/workflows/team-memory-coordinator.yml | 2 +- .github/workflows/team-memory-post-merge.yml | 16 ++-- .github/workflows/team-memory-tests.yml | 8 +- 7 files changed, 126 insertions(+), 172 deletions(-) delete mode 100644 .github/actions/queue-team-memory/action.yml diff --git a/.github/actions/queue-team-memory/README.md b/.github/actions/queue-team-memory/README.md index 6bd6e48c..3578e27c 100644 --- a/.github/actions/queue-team-memory/README.md +++ b/.github/actions/queue-team-memory/README.md @@ -5,6 +5,12 @@ The [push caller](../../workflows/team-memory-post-merge.yml) only preserves source identities and dispatches the [coordinator](../../workflows/team-memory-coordinator.yml). Manual reconciliation now runs through the coordinator too, not the old push workflow. No source caller logs in to Azure or invokes the agent. +The caller reuses `microsoft/IssueLens/.github/actions/queue-team-memory`; +this directory contains only Java coordinator validation, setup guidance, and +contract tests, not a second dispatch action. +Its prerequisite is [microsoft/IssueLens#51](https://github.com/microsoft/IssueLens/pull/51), +pinned at `521b1ff80a70140e286ced7500d0b150cf9e6d6c`. Review and merge that +prerequisite first; these repository changes need no hosted runtime deployment. Both workflows require **`ISSUELENS_TEAM_MEMORY_COORDINATOR_ENABLED=true`**. This is a new, disabled-by-default repository variable: the existing @@ -33,14 +39,14 @@ public visibility, and default branches. These sources currently configure The two Spring repositories are not included: they currently have no `.github/issuelens.yml` policy. An allowlist entry is not a source migration or authorization to ignore policy. Only Java Pack's caller changes in this PR. -Other repositories must adopt this action in separate reviewed PRs. +Other repositories must adopt the shared IssueLens action in separate reviewed PRs. Renames, transfers, branch changes, or private sources require another review. The hosted agent must resolve each source's policy and privacy constraints, verify that its validated destination is this wiki, and stop rather than override a conflicting policy. Requests explicitly require this destination; the pinned client's final receipt validator rejects any other wiki. Neither -the dispatch action nor its allowlist grants wiki write access. +the shared dispatch action nor the coordinator's allowlist grants wiki write access. ## Future external callers @@ -51,16 +57,23 @@ needed for a remote reference: ```yaml - name: Queue Java team-memory update - uses: microsoft/vscode-java-pack/.github/actions/queue-team-memory@FULL_REVIEWED_COMMIT_SHA + uses: microsoft/IssueLens/.github/actions/queue-team-memory@521b1ff80a70140e286ced7500d0b150cf9e6d6c with: + coordinator-repository: microsoft/vscode-java-pack + coordinator-workflow: team-memory-coordinator.yml + coordinator-ref: main source-token: ${{ github.token }} dispatch-token: ${{ steps.dispatch-token.outputs.token }} ``` -Replace the placeholder with the reviewed, merged 40-character revision of -this action; do not use a moving branch. The caller must obtain +The shared IssueLens action is pinned to the immutable prerequisite revision. +Do not use a moving branch. All three coordinator inputs must be supplied +together. `coordinator-ref: main` is the destination +branch, independently of a source default branch such as `develop`. +The caller must obtain `steps.dispatch-token.outputs.token` using a **dedicated dispatch GitHub App**, -restricted to Java Pack with **Actions write**. The read and dispatch tokens +restricted to Java Pack with **Actions write and Contents read** (the latter +validates the target branch). The read and dispatch tokens are separate inputs. An explicitly empty token fails; it never falls back. Do not distribute the hosted IssueLens App's private key to source workflows. A source repository's `GITHUB_TOKEN` remains repository-scoped regardless of @@ -107,16 +120,20 @@ default-branch push from the exact source workflow. Its artifact contains repository/run/workflow identities, before/after SHAs, and commit IDs only: no commit messages, bodies, code, credentials, or agent output. It is capped at 64 KiB and 1,000 commits and retained for seven days. +Before its single POST, the shared action revalidates the source and uses the +dispatch token to validate the target repository, exact active workflow path, +and independently selected target branch/full SHA. Before downloading, the coordinator authenticates source run/attempt and artifact metadata, repository IDs, workflow, branch, actors, digest, expiry, and size. Pinned artifact download enforces the digest. Preflight repeats the metadata checks, validates the downloaded identity-only schema, and reuses IssueLens's complete bounded push/PR discovery and merge revalidation. The -client is pinned to the actual merged revision of -[microsoft/IssueLens#50](https://github.com/microsoft/IssueLens/pull/50), -`4175ea71e170938826fb847e6bd5108f0f5597cf`. Only the small Java provenance -adapter is local; request, stream, discovery, and receipt machinery is not +client and dispatch action are pinned to the same shared-dispatcher prerequisite +revision, `521b1ff80a70140e286ced7500d0b150cf9e6d6c`, built on the merged +[microsoft/IssueLens#50](https://github.com/microsoft/IssueLens/pull/50). +Only the small Java provenance adapter is local; request, stream, discovery, +and receipt machinery is not copied or monkey-patched. The pilot's IssueLens-only coordinator adapter is deliberately not called. @@ -142,5 +159,7 @@ With the pinned IssueLens action files available locally, set python -m unittest discover -s .github/actions/queue-team-memory/tests ``` -The focused CI workflow loads the same immutable client revision. These checks -use fake API/stream responses and make no dispatch, Azure, or wiki calls. +The focused CI workflow loads the same immutable client and shared-action +revision. These checks pass the shared dispatcher's actual snapshot through +Java preflight and use fake API/stream responses; they make no dispatch, Azure, +or wiki calls. diff --git a/.github/actions/queue-team-memory/action.yml b/.github/actions/queue-team-memory/action.yml deleted file mode 100644 index 07ca85e8..00000000 --- a/.github/actions/queue-team-memory/action.yml +++ /dev/null @@ -1,46 +0,0 @@ -name: Queue Java team memory -description: Preserve an allowlisted default-branch push and dispatch the central Java wiki coordinator. - -inputs: - source-token: - description: Source repository read token for push provenance validation. - required: false - default: ${{ github.token }} - dispatch-token: - description: Token with Actions write access to microsoft/vscode-java-pack. - required: false - default: ${{ github.token }} - -outputs: - source-artifact-id: - description: Uploaded identity artifact ID; not proof of dispatch acceptance or successful maintenance. - value: ${{ steps.artifact.outputs.artifact-id }} - -runs: - using: composite - steps: - - name: Prepare identity-only source event - id: source - shell: bash - env: - GITHUB_ACTION_PATH: ${{ github.action_path }} - GH_TOKEN: ${{ inputs.source-token }} - run: python3 -I "$GITHUB_ACTION_PATH/team_memory.py" prepare-dispatch - - - name: Preserve source event for the coordinator - id: artifact - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 - with: - name: issuelens-team-memory-source-${{ github.run_attempt }} - path: ${{ steps.source.outputs.source-event-path }} - if-no-files-found: error - retention-days: 7 - - - name: Queue one team-memory request - shell: bash - env: - GITHUB_ACTION_PATH: ${{ github.action_path }} - GH_TOKEN: ${{ inputs.source-token }} - DISPATCH_TOKEN: ${{ inputs.dispatch-token }} - SOURCE_ARTIFACT_ID: ${{ steps.artifact.outputs.artifact-id }} - run: python3 -I "$GITHUB_ACTION_PATH/team_memory.py" dispatch diff --git a/.github/actions/queue-team-memory/team_memory.py b/.github/actions/queue-team-memory/team_memory.py index 78888a5c..e7875094 100644 --- a/.github/actions/queue-team-memory/team_memory.py +++ b/.github/actions/queue-team-memory/team_memory.py @@ -1,4 +1,4 @@ -"""Identity-only Java dispatch and cross-repository adapter for the pinned IssueLens client.""" +"""Java coordinator provenance adapter for the pinned IssueLens client.""" import argparse import importlib.util @@ -15,7 +15,6 @@ COORDINATOR_WORKFLOW = ".github/workflows/team-memory-coordinator.yml" SOURCE_INPUTS = ("SOURCE_RUN_ID", "SOURCE_RUN_ATTEMPT", "SOURCE_ARTIFACT_ID") MAX_SOURCE_BYTES = 64 * 1024 -MAX_PUSH_COMMITS = 1000 # Reviewed public sources with an existing policy targeting the Java Pack wiki. SOURCES = { COORDINATOR: (104967329, "main"), @@ -72,18 +71,14 @@ def redirect_request(self, *args, **kwargs): return None -def github_request(path, token, payload=None): +def github_read(path, token): require(isinstance(token, str) and token.strip(), "Required GitHub token is empty") - return urllib.request.Request( + request = urllib.request.Request( "https://api.github.com" + path, - data=None if payload is None else json.dumps(payload).encode("utf-8"), headers={"Authorization": "Bearer " + token, "Accept": "application/vnd.github+json", "Content-Type": "application/json", "X-GitHub-Api-Version": "2022-11-28"}, ) - - -def github_read(path, token): - with urllib.request.build_opener(NoRedirect()).open(github_request(path, token), timeout=30) as response: + with urllib.request.build_opener(NoRedirect()).open(request, timeout=30) as response: content = response.read(4 * 1024 * 1024 + 1) require(len(content) <= 4 * 1024 * 1024, "GitHub response exceeds its size limit") result = json.loads(content, object_pairs_hook=unique_object) @@ -118,22 +113,6 @@ def workflow_context(repository, workflow, event_name, token): return current, event -def validate_push(event, reference, head): - require(all(event.get(flag) is False for flag in ("created", "deleted", "forced")), - "Created, deleted, or forced refs require manual PR reconciliation") - before, after = full_sha(event.get("before")), full_sha(event.get("after")) - require(before != after and event.get("ref") == reference and head == after - and isinstance(event.get("head_commit"), dict) and event["head_commit"].get("id") == after, - "Push source identity mismatch") - commits = event.get("commits") - require(isinstance(commits, list) and 0 < len(commits) <= MAX_PUSH_COMMITS - and all(isinstance(item, dict) for item in commits), "Missing or oversized push inventory") - shas = [full_sha(item.get("id")) for item in commits] - require(len(set(shas)) == len(shas) and after in shas and before not in shas, - "Push inventory has duplicate or mismatched identities") - return before, after, shas - - def source_event_path(): return Path(os.environ["RUNNER_TEMP"]) / "issuelens-team-memory-source" / "source-event.json" @@ -144,62 +123,6 @@ def outputs(values): output.write(f"{name}={value}\n") -def push_snapshot(): - repository = os.environ["GITHUB_REPOSITORY"] - current, event = workflow_context(repository, SOURCE_WORKFLOW, "push", os.environ["GH_TOKEN"]) - before, after, shas = validate_push(event, os.environ["GITHUB_REF"], os.environ["GITHUB_SHA"]) - require(os.environ["GITHUB_WORKFLOW_SHA"] == after, "Source workflow revision does not match the push") - actors = [os.environ[name] for name in ("GITHUB_ACTOR", "GITHUB_TRIGGERING_ACTOR")] - require(all(re.fullmatch(r"[A-Za-z0-9-]+(?:\[bot\])?", actor) for actor in actors), "Invalid source actors") - return { - "metadata": { - "repository": repository, "repository_id": current["id"], "base_ref": current["default_branch"], - "event_name": "push", "event_action": "push", - "actor_login": actors[0], "triggering_actor": actors[1], - "workflow_ref": os.environ["GITHUB_WORKFLOW_REF"], "workflow_sha": after, - "run_id": positive(os.environ["GITHUB_RUN_ID"]), "run_attempt": positive(os.environ["GITHUB_RUN_ATTEMPT"]), - }, - "event": { - "repository": {"id": current["id"], "full_name": repository}, "ref": event["ref"], - "before": before, "after": after, "created": False, "deleted": False, "forced": False, - "commits": [{"id": sha} for sha in shas], "head_commit": {"id": after}, - }, - } - - -def prepare_dispatch(): - snapshot = push_snapshot() - content = json.dumps(snapshot, separators=(",", ":")).encode("utf-8") - require(len(content) <= MAX_SOURCE_BYTES, "Source event exceeds 64 KiB") - path = source_event_path() - path.parent.mkdir(exist_ok=True) - path.write_bytes(content) - outputs({"source-event-path": path}) - print("Prepared identity-only team-memory source event") - - -def dispatch(): - token = os.environ["DISPATCH_TOKEN"] - require(token.strip(), "dispatch-token must be non-empty") - snapshot = read_json(source_event_path(), MAX_SOURCE_BYTES) - expected = push_snapshot() - require(snapshot == expected and matching_metadata(snapshot["metadata"], expected["metadata"]), - "Source event changed before dispatch") - central = project(COORDINATOR, token) - metadata = snapshot["metadata"] - request = github_request( - f"/repos/{COORDINATOR}/actions/workflows/team-memory-coordinator.yml/dispatches", token, - {"ref": central["default_branch"], "inputs": { - "source_repository": metadata["repository"], "source_run_id": str(metadata["run_id"]), - "source_run_attempt": str(metadata["run_attempt"]), - "source_artifact_id": str(positive(os.environ["SOURCE_ARTIFACT_ID"])), - }}, - ) - with urllib.request.build_opener(NoRedirect()).open(request, timeout=30) as response: - require(response.status in {200, 204}, "Dispatch was not acknowledged; do not retry blindly") - print("Coordinator dispatch accepted; maintenance completion is reported by the coordinator run") - - def selection(): repository = os.environ.get("SOURCE_REPOSITORY", "") require(repository in SOURCES, "Supply an allowlisted source_repository") @@ -310,7 +233,6 @@ def preflight(): and all(isinstance(item, dict) and set(item) == {"id"} for item in push["commits"]) and push["head_commit"] == {"id": metadata["workflow_sha"]}, "Invalid identity-only push artifact") - validate_push(push, "refs/heads/" + current["default_branch"], metadata["workflow_sha"]) try: envelope = client.prepare_push_memory(repository, current, push, {**metadata, **central}) except client.SkippedRequest as skipped: @@ -332,17 +254,14 @@ def preflight(): def run(command): try: - {"prepare-dispatch": prepare_dispatch, "dispatch": dispatch, "select-source": select_source, - "validate-dispatch": validate_dispatch, "preflight": preflight}[command]() + {"select-source": select_source, "validate-dispatch": validate_dispatch, "preflight": preflight}[command]() except ValueError as error: raise SystemExit(f"::error::{error}") from None except Exception: - message = ("Coordinator dispatch failed or its outcome is unknown; inspect coordinator runs before retrying" - if command == "dispatch" else "Java team-memory validation failed; no agent request was sent") - raise SystemExit(f"::error::{message}") from None + raise SystemExit("::error::Java team-memory validation failed; no agent request was sent") from None if __name__ == "__main__": parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("command", choices=("prepare-dispatch", "dispatch", "select-source", "validate-dispatch", "preflight")) + parser.add_argument("command", choices=("select-source", "validate-dispatch", "preflight")) run(parser.parse_args().command) diff --git a/.github/actions/queue-team-memory/tests/test_team_memory.py b/.github/actions/queue-team-memory/tests/test_team_memory.py index 02c4165c..af8563f9 100644 --- a/.github/actions/queue-team-memory/tests/test_team_memory.py +++ b/.github/actions/queue-team-memory/tests/test_team_memory.py @@ -3,6 +3,7 @@ import io import json import os +import re import tempfile import unittest from datetime import datetime, timedelta, timezone @@ -47,6 +48,8 @@ def setUp(self): self.before, self.merge, self.after, self.tip = [letter * 40 for letter in "daef"] self.project = self.make_project(self.repository) self.central = self.make_project(action.COORDINATOR) + self.coordinator_workflow = {"id": 789, "path": action.COORDINATOR_WORKFLOW, "state": "active"} + self.coordinator_branch = {"name": "main", "commit": {"sha": self.tip}} self.environment = { "GITHUB_REPOSITORY": action.COORDINATOR, "GITHUB_EVENT_NAME": "workflow_dispatch", "GITHUB_REF": "refs/heads/main", "GITHUB_SHA": self.tip, "GITHUB_WORKFLOW_SHA": self.tip, @@ -57,6 +60,8 @@ def setUp(self): "GITHUB_OUTPUT": str(self.directory / "output.txt"), "COORDINATOR_TOKEN": "test-coordinator-token", "GH_TOKEN": "test-source-token", "DISPATCH_TOKEN": "test-dispatch-token", "SOURCE_REPOSITORY": self.repository, + "COORDINATOR_REPOSITORY": action.COORDINATOR, + "COORDINATOR_WORKFLOW": "team-memory-coordinator.yml", "COORDINATOR_REF": "main", "SOURCE_RUN_ID": "123456", "SOURCE_RUN_ATTEMPT": "2", "SOURCE_ARTIFACT_ID": "456", "DISPATCH_PR": "", "OUTPUT_MODE": "quiet", "SUMMARY_MODE": "none", "AGENT_URL": "https://test.services.ai.azure.com/agents/test/protocols/invocations", @@ -117,8 +122,8 @@ def setUp(self): self.api_reads = [] self.mock_read = patch.object(action, "github_read", side_effect=self.api_read).start() self.addCleanup(patch.stopall) - patch.object(self.client, "github_read", side_effect=lambda path, payload=None: self.api_read( - path, os.environ["GH_TOKEN"], payload)).start() + patch.object(self.client, "github_read", side_effect=lambda path, payload=None, **kwargs: self.api_read( + path, kwargs.get("token") or os.environ["GH_TOKEN"], payload)).start() patch.object(action, "load_client", return_value=self.client).start() self.opener = Mock() self.opener.open.return_value = Response() @@ -138,6 +143,8 @@ def api_read(self, path, token, payload=None): f"/repos/{self.repository}/actions/artifacts/456": self.artifact, f"/repos/{self.repository}/pulls/27": self.pull, f"/repos/{self.repository}/compare/{self.before}...{self.after}?per_page=1&page=2": self.comparison, + f"/repos/{action.COORDINATOR}/actions/workflows/team-memory-coordinator.yml": self.coordinator_workflow, + f"/repos/{action.COORDINATOR}/branches/main": self.coordinator_branch, "/graphql": self.graphql, } self.assertIn(path, responses, "Unexpected API access") @@ -168,7 +175,10 @@ def select_push(self): self.write_event(self.push) def execute(self, command): - action.run(command) + if command in {"prepare-dispatch", "dispatch"}: + self.client.run(command) + else: + action.run(command) self.azure.assert_not_called() def test_identity_only_source_excludes_all_untrusted_text_and_tokens(self): @@ -184,8 +194,26 @@ def test_identity_only_source_excludes_all_untrusted_text_and_tokens(self): self.assertNotIn("test-source-token", path.read_text()) self.opener.open.assert_not_called() + def test_shared_dispatcher_snapshot_passes_java_preflight_without_translation(self): + coordinator_environment = dict(os.environ) + self.select_push() + self.execute("prepare-dispatch") + self.assertEqual(json.loads(action.source_event_path().read_bytes()), self.snapshot) + Path(os.environ["GITHUB_OUTPUT"]).unlink() + os.environ.clear() + os.environ.update(coordinator_environment) + self.write_event({"repository": self.central}) + self.execute("validate-dispatch") + Path(os.environ["GITHUB_OUTPUT"]).unlink() + self.execute("preflight") + metadata = self.envelope()["metadata"] + self.assertEqual(metadata["repository"], self.repository) + self.assertEqual(metadata["push_after"], self.after) + self.assertEqual(metadata["required_wiki_repository"], action.COORDINATOR) + self.opener.open.assert_not_called() + def test_maximum_inventory_fits_and_matches_upstream_limits(self): - self.assertEqual(action.MAX_PUSH_COMMITS, self.client.MAX_PUSH_COMMITS) + self.assertEqual(self.client.MAX_PUSH_COMMITS, 1000) self.assertEqual(action.MAX_SOURCE_BYTES, self.client.MAX_SOURCE_BYTES) self.select_push() event = copy.deepcopy(self.push) @@ -201,10 +229,11 @@ def test_every_allowlisted_source_uses_its_own_identity_and_default_branch(self) **original, "GITHUB_REPOSITORY": repository, "GITHUB_EVENT_NAME": "push", "GITHUB_REF": "refs/heads/" + branch, "GITHUB_SHA": self.after, "GITHUB_WORKFLOW_SHA": self.after, "GITHUB_WORKFLOW_REF": repository + "/" + action.SOURCE_WORKFLOW + "@refs/heads/" + branch, - }, clear=True), patch.object(action, "github_read", return_value=self.make_project(repository)): + }, clear=True), patch.object(self.client, "github_read", return_value=self.make_project(repository)): self.write_event({**self.push, "repository": {"id": identity, "full_name": repository}, "ref": "refs/heads/" + branch}) - snapshot = action.push_snapshot() + self.client.prepare_dispatch() + snapshot = json.loads(action.source_event_path().read_bytes()) self.assertEqual(snapshot["metadata"]["repository_id"], identity) self.assertEqual(snapshot["metadata"]["base_ref"], branch) self.assertEqual(snapshot["event"]["repository"]["full_name"], repository) @@ -225,13 +254,13 @@ def test_push_rejects_unsafe_or_incomplete_identities(self): self.assertFalse(action.source_event_path().exists()) self.assertEqual(self.output(), {}) - def test_push_rejects_wrong_workflow_branch_revision_and_repository(self): + def test_push_rejects_wrong_workflow_branch_revision_and_event(self): self.select_push() original = dict(os.environ) for name, value in ( ("GITHUB_WORKFLOW_REF", self.repository + "/.github/workflows/untrusted.yml@refs/heads/develop"), ("GITHUB_REF", "refs/heads/main"), ("GITHUB_WORKFLOW_SHA", self.tip), - ("GITHUB_EVENT_NAME", "pull_request"), ("GITHUB_REPOSITORY", "microsoft/vscode-spring-initializr"), + ("GITHUB_EVENT_NAME", "pull_request"), ): with self.subTest(name=name), patch.dict(os.environ, {**original, name: value}, clear=True): with self.assertRaises(SystemExit): @@ -245,6 +274,8 @@ def test_dispatch_sends_one_post_to_central_main_with_separate_tokens(self): self.assertEqual([(path, token) for path, token, _ in self.api_reads], [ (f"/repos/{self.repository}", "test-source-token"), (f"/repos/{action.COORDINATOR}", "test-dispatch-token"), + (f"/repos/{action.COORDINATOR}/actions/workflows/team-memory-coordinator.yml", "test-dispatch-token"), + (f"/repos/{action.COORDINATOR}/branches/main", "test-dispatch-token"), ]) self.opener.open.assert_called_once() request = self.opener.open.call_args.args[0] @@ -277,11 +308,31 @@ def test_explicit_empty_dispatch_token_fails_before_any_network_access(self): self.mock_read.assert_not_called() self.opener.open.assert_not_called() - def test_dispatch_revalidates_the_original_event_not_only_saved_metadata(self): + def test_dispatch_revalidates_saved_push_inventory_before_target_access(self): self.select_push() self.snapshot["event"]["before"] = self.merge self.write_source() - with self.assertRaisesRegex(SystemExit, "Source event changed"): + with self.assertRaisesRegex(SystemExit, "Push commit inventory"): + self.execute("dispatch") + self.opener.open.assert_not_called() + self.assertFalse(any(token == "test-dispatch-token" for _, token, _ in self.api_reads)) + + def test_shared_dispatch_rejects_inactive_or_wrong_target_workflow_without_post(self): + self.select_push() + self.write_source() + original = copy.deepcopy(self.coordinator_workflow) + for changes in ({"state": "disabled_manually"}, {"path": ".github/workflows/untrusted.yml"}): + with self.subTest(changes=changes): + self.coordinator_workflow = {**original, **changes} + with self.assertRaisesRegex(SystemExit, "Coordinator workflow identity"): + self.execute("dispatch") + self.opener.open.assert_not_called() + + def test_shared_dispatch_rejects_wrong_target_branch_without_post(self): + self.select_push() + self.write_source() + self.coordinator_branch["name"] = "develop" + with self.assertRaisesRegex(SystemExit, "Coordinator branch identity"): self.execute("dispatch") self.opener.open.assert_not_called() @@ -584,12 +635,18 @@ def test_only_verified_sources_are_allowlisted(self): def test_workflows_share_new_opt_in_and_only_coordinator_invokes(self): source = (ROOT / ".github/workflows/team-memory-post-merge.yml").read_text() central = (ROOT / ".github/workflows/team-memory-coordinator.yml").read_text() - composite = (ACTION / "action.yml").read_text() + shared = Path(os.environ["ISSUELENS_CLIENT_PATH"]).resolve().parent.parent / "queue-team-memory" / "action.yml" + composite = shared.read_text() + self.assertFalse((ACTION / "action.yml").exists()) + for command in ("prepare_dispatch", "dispatch", "push_snapshot"): + self.assertFalse(hasattr(action, command), "Source dispatch belongs only to the shared action") for workflow in (source, central): self.assertIn("ISSUELENS_TEAM_MEMORY_COORDINATOR_ENABLED == 'true'", workflow) self.assertNotIn("ISSUELENS_TEAM_MEMORY_ENABLED", workflow) self.assertIn("permissions: {}", workflow) - self.assertIn("persist-credentials: false", workflow) + self.assertNotIn("actions/checkout@", source) + self.assertNotIn("python", source) + self.assertIn("persist-credentials: false", central) self.assertNotIn("concurrency:", source) self.assertNotIn("workflow_dispatch:", source) for text in (source, composite): @@ -607,9 +664,14 @@ def test_workflows_share_new_opt_in_and_only_coordinator_invokes(self): self.assertIn("retention-days: 7", composite) self.assertIn("GH_TOKEN: ${{ inputs.source-token }}", composite) self.assertIn("DISPATCH_TOKEN: ${{ inputs.dispatch-token }}", composite) + for name, value in (("repository", action.COORDINATOR), ("workflow", "team-memory-coordinator.yml"), ("ref", "main")): + self.assertIn(f"coordinator-{name}: {value}", source) def test_client_pin_matches_coordinator_tests_and_documentation(self): - revision = "4175ea71e170938826fb847e6bd5108f0f5597cf" + source = (ROOT / ".github/workflows/team-memory-post-merge.yml").read_text() + match = re.search(r"uses: microsoft/IssueLens/\.github/actions/queue-team-memory@([0-9a-f]{40})", source) + self.assertIsNotNone(match) + revision = match.group(1) for relative in (".github/workflows/team-memory-coordinator.yml", ".github/workflows/team-memory-tests.yml"): self.assertIn("ref: " + revision, (ROOT / relative).read_text()) self.assertIn(revision, (ACTION / "README.md").read_text()) diff --git a/.github/workflows/team-memory-coordinator.yml b/.github/workflows/team-memory-coordinator.yml index cf444740..be602309 100644 --- a/.github/workflows/team-memory-coordinator.yml +++ b/.github/workflows/team-memory-coordinator.yml @@ -69,7 +69,7 @@ jobs: uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: repository: microsoft/IssueLens - ref: 4175ea71e170938826fb847e6bd5108f0f5597cf # merged PR 50 + ref: 521b1ff80a70140e286ced7500d0b150cf9e6d6c # shared dispatcher prerequisite path: .issuelens-client persist-credentials: false sparse-checkout: /.github/actions/issuelens/ diff --git a/.github/workflows/team-memory-post-merge.yml b/.github/workflows/team-memory-post-merge.yml index e9295e71..065f4ef1 100644 --- a/.github/workflows/team-memory-post-merge.yml +++ b/.github/workflows/team-memory-post-merge.yml @@ -19,13 +19,11 @@ jobs: contents: read actions: write steps: - - name: Load dispatcher from trusted push revision - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - ref: ${{ github.workflow_sha }} - persist-credentials: false - sparse-checkout: /.github/actions/queue-team-memory/ - sparse-checkout-cone-mode: false - - name: Queue team-memory update - uses: ./.github/actions/queue-team-memory + uses: microsoft/IssueLens/.github/actions/queue-team-memory@521b1ff80a70140e286ced7500d0b150cf9e6d6c + with: + coordinator-repository: microsoft/vscode-java-pack + coordinator-workflow: team-memory-coordinator.yml + coordinator-ref: main + source-token: ${{ github.token }} + dispatch-token: ${{ github.token }} diff --git a/.github/workflows/team-memory-tests.yml b/.github/workflows/team-memory-tests.yml index cb9e2c85..0959b2ef 100644 --- a/.github/workflows/team-memory-tests.yml +++ b/.github/workflows/team-memory-tests.yml @@ -23,14 +23,16 @@ jobs: with: persist-credentials: false - - name: Load reviewed IssueLens client + - name: Load reviewed IssueLens client and shared dispatcher uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: repository: microsoft/IssueLens - ref: 4175ea71e170938826fb847e6bd5108f0f5597cf + ref: 521b1ff80a70140e286ced7500d0b150cf9e6d6c path: .issuelens-client persist-credentials: false - sparse-checkout: /.github/actions/issuelens/ + sparse-checkout: | + /.github/actions/issuelens/ + /.github/actions/queue-team-memory/ sparse-checkout-cone-mode: false - name: Check source trust, dispatch, discovery, and receipts offline From f1e14a306a7f72f329e14499009a7ef8543ea7ac Mon Sep 17 00:00:00 2001 From: Changyong Gong Date: Sat, 10 Oct 2026 11:03:10 +0800 Subject: [PATCH 3/5] Use finalized shared IssueLens team-memory actions Replace artifact provenance and the local coordinator adapter with the pinned generic dispatcher and shared invocation action. Preserve trusted source IDs, scoped read authentication, the central wiki queue, and validated reconciliation receipts. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- .github/actions/queue-team-memory/README.md | 134 ++- .../actions/queue-team-memory/team_memory.py | 267 ----- .../tests/test_team_memory.py | 1029 +++++++---------- .github/workflows/team-memory-coordinator.yml | 120 +- .github/workflows/team-memory-post-merge.yml | 17 +- .github/workflows/team-memory-tests.yml | 10 +- 6 files changed, 588 insertions(+), 989 deletions(-) delete mode 100644 .github/actions/queue-team-memory/team_memory.py diff --git a/.github/actions/queue-team-memory/README.md b/.github/actions/queue-team-memory/README.md index 3578e27c..6341c37e 100644 --- a/.github/actions/queue-team-memory/README.md +++ b/.github/actions/queue-team-memory/README.md @@ -1,16 +1,19 @@ # Java team-memory coordinator Java Pack hosts the single queue for the `microsoft/vscode-java-pack` wiki. -The [push caller](../../workflows/team-memory-post-merge.yml) only preserves -source identities and dispatches the [coordinator](../../workflows/team-memory-coordinator.yml). -Manual reconciliation now runs through the coordinator too, not the old push -workflow. No source caller logs in to Azure or invokes the agent. -The caller reuses `microsoft/IssueLens/.github/actions/queue-team-memory`; -this directory contains only Java coordinator validation, setup guidance, and -contract tests, not a second dispatch action. -Its prerequisite is [microsoft/IssueLens#51](https://github.com/microsoft/IssueLens/pull/51), -pinned at `521b1ff80a70140e286ced7500d0b150cf9e6d6c`. Review and merge that -prerequisite first; these repository changes need no hosted runtime deployment. +The [push caller](../../workflows/team-memory-post-merge.yml) invokes only the +standalone IssueLens dispatch action with five scalar reconciliation inputs. +The [coordinator](../../workflows/team-memory-coordinator.yml) invokes the shared +IssueLens invocation action directly. Both actions are pinned to current upstream +`a81d2d96167fc0e69ac631c2edc85f858e693289`, including the merged shared-action work +in [microsoft/IssueLens#51](https://github.com/microsoft/IssueLens/pull/51). +There is no unmerged prerequisite or hosted runtime deployment for action reuse. + +Manual PR reconciliation also runs through this central queue. There is no +source preparation script, checkout, artifact upload/download, or local +invoker/provenance adapter. This directory retains setup guidance and offline +contract tests only. Shared actions own transport, source validation, discovery, +Azure OIDC login, invocation, and final receipt checks. Both workflows require **`ISSUELENS_TEAM_MEMORY_COORDINATOR_ENABLED=true`**. This is a new, disabled-by-default repository variable: the existing @@ -21,8 +24,11 @@ enablement. Issue-loop and direct-task consumers are unchanged. ## Source trust and scope -The reviewed allowlist in `team_memory.py` binds names, numeric repository IDs, -public visibility, and default branches. These sources currently configure +The trusted `ISSUELENS_SOURCE_REPOSITORIES` JSON in the coordinator workflow maps +canonical source names to immutable numeric repository IDs. It is never supplied +by a dispatch input. The shared invocation action validates canonical identity, +public-to-public cross-repository access, and each source's current default +branch against its source run/PR. These sources currently configure `instructions.team_memory.wiki_repository: microsoft/vscode-java-pack`: | Source | Default branch | Reviewed policy snapshot | @@ -36,8 +42,8 @@ public visibility, and default branches. These sources currently configure | `microsoft/java-debug` | `main` | [bcae605a](https://github.com/microsoft/java-debug/blob/bcae605a6240f11b447163a614e215721c11b107/.github/issuelens.yml) | | `microsoft/build-server-for-gradle` | `develop` | [ab37cca9](https://github.com/microsoft/build-server-for-gradle/blob/ab37cca97693c60e90d257ce9eb6e9575da7e6f1/.github/issuelens.yml) | -The two Spring repositories are not included: they currently have no -`.github/issuelens.yml` policy. An allowlist entry is not a source migration or +The two Spring candidates remain excluded: their policies were absent/unverified. +An allowlist entry is not a source migration or authorization to ignore policy. Only Java Pack's caller changes in this PR. Other repositories must adopt the shared IssueLens action in separate reviewed PRs. Renames, transfers, branch changes, or private sources require another review. @@ -46,7 +52,7 @@ The hosted agent must resolve each source's policy and privacy constraints, verify that its validated destination is this wiki, and stop rather than override a conflicting policy. Requests explicitly require this destination; the pinned client's final receipt validator rejects any other wiki. Neither -the shared dispatch action nor the coordinator's allowlist grants wiki write access. +the standalone dispatch action nor the coordinator's allowlist grants wiki write access. ## Future external callers @@ -57,28 +63,37 @@ needed for a remote reference: ```yaml - name: Queue Java team-memory update - uses: microsoft/IssueLens/.github/actions/queue-team-memory@521b1ff80a70140e286ced7500d0b150cf9e6d6c + uses: microsoft/IssueLens/.github/actions/queue-team-memory@a81d2d96167fc0e69ac631c2edc85f858e693289 with: coordinator-repository: microsoft/vscode-java-pack coordinator-workflow: team-memory-coordinator.yml coordinator-ref: main - source-token: ${{ github.token }} dispatch-token: ${{ steps.dispatch-token.outputs.token }} + workflow-inputs: >- + {"source_repository":"${{ github.repository }}", + "source_run_id":"${{ github.run_id }}", + "source_run_attempt":"${{ github.run_attempt }}", + "push_before":"${{ github.event.before }}", + "push_after":"${{ github.sha }}"} ``` -The shared IssueLens action is pinned to the immutable prerequisite revision. -Do not use a moving branch. All three coordinator inputs must be supplied -together. `coordinator-ref: main` is the destination -branch, independently of a source default branch such as `develop`. +Do not use a moving action branch. `coordinator-ref: main` is the destination +branch, independently of a source default branch such as `develop`. Source +workflows must gate on the actual default branch, `github.workflow_sha == +github.sha`, and non-created/non-deleted/non-forced pushes, as Java Pack's caller +does. The generic action does not infer or attest source fields; the caller owns +the JSON input mapping. + The caller must obtain `steps.dispatch-token.outputs.token` using a **dedicated dispatch GitHub App**, restricted to Java Pack with **Actions write and Contents read** (the latter -validates the target branch). The read and dispatch tokens -are separate inputs. An explicitly empty token fails; it never falls back. +validates the target branch). The source-read token belongs only to the receiving +invocation action, not the generic dispatch action. An explicitly empty +dispatch token fails; it never falls back. Do not distribute the hosted IssueLens App's private key to source workflows. A source repository's `GITHUB_TOKEN` remains repository-scoped regardless of -its `actions: write` permission. Java Pack's own caller can use its token for -both inputs without new credentials. +its `actions: write` permission. Java Pack's own caller uses its token for the +target dispatch without new credentials. ## Coordinator prerequisites @@ -91,8 +106,10 @@ both inputs without new credentials. Pull requests read**. Configure central secrets `ISSUELENS_SOURCE_READ_APP_CLIENT_ID` and `ISSUELENS_SOURCE_READ_APP_PRIVATE_KEY`. The workflow mints an ephemeral, - single-source token only after validating source selection and revokes it at - job end. Java Pack requests use its own `GITHUB_TOKEN` instead. Source-read + single-source token only after checking the trusted source-name map and revokes + it at job end. The invocation action independently validates source identity, + run/head, current branch ancestry, and the complete range before Azure login. + Java Pack requests use its own `GITHUB_TOKEN` instead. Source-read and dispatch installations may differ and must mint separate tokens. - The existing hosted IssueLens identity must independently have permission to read the source and publish to the Java Pack wiki, with its normal @@ -100,14 +117,16 @@ both inputs without new credentials. its source-code/issue write permissions for this feature. The new App secrets are setup prerequisites, not assumed existing credentials. -No live invocation is required to validate these repository changes. +Verify the central default-branch `workflow_dispatch` Azure OIDC subject and +endpoint access, not just the old source-push identity. No live invocation is +required to validate these repository changes. ## Verification and outcomes -Automatic dispatch requires explicit `source_repository`, `source_run_id`, -`source_run_attempt`, and `source_artifact_id`. Manual dispatch supplies -`source_repository` and one `pull_request_number`, with all three automatic -identifiers empty. Mixed or incomplete requests fail before source access. +Range reconciliation supplies `source_repository`, `source_run_id`, +`source_run_attempt`, `push_before`, and `push_after`. Manual dispatch supplies +`source_repository` and one `pull_request_number`, with all four range fields +empty. Mixed or incomplete requests fail before invocation. All runs use the same fixed wiki concurrency group, `queue: max`, and `cancel-in-progress: false`, including validation, discovery, invocation, and final receipt verification. GitHub supports one active run and at most 100 @@ -115,27 +134,21 @@ pending runs in this queue; it is not an unlimited or guaranteed-delivery queue. See GitHub's [concurrency documentation](https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/control-workflow-concurrency) for the `queue: max` contract. -The dispatcher accepts only a trusted, non-created/non-deleted/non-forced -default-branch push from the exact source workflow. Its artifact contains -repository/run/workflow identities, before/after SHAs, and commit IDs only: -no commit messages, bodies, code, credentials, or agent output. It is capped -at 64 KiB and 1,000 commits and retained for seven days. -Before its single POST, the shared action revalidates the source and uses the -dispatch token to validate the target repository, exact active workflow path, -and independently selected target branch/full SHA. - -Before downloading, the coordinator authenticates source run/attempt and -artifact metadata, repository IDs, workflow, branch, actors, digest, expiry, -and size. Pinned artifact download enforces the digest. Preflight repeats the -metadata checks, validates the downloaded identity-only schema, and reuses -IssueLens's complete bounded push/PR discovery and merge revalidation. The -client and dispatch action are pinned to the same shared-dispatcher prerequisite -revision, `521b1ff80a70140e286ced7500d0b150cf9e6d6c`, built on the merged -[microsoft/IssueLens#50](https://github.com/microsoft/IssueLens/pull/50). -Only the small Java provenance adapter is local; request, stream, discovery, -and receipt machinery is not -copied or monkey-patched. The pilot's IssueLens-only coordinator adapter is -deliberately not called. +The standalone dispatcher validates only the target repository, active workflow, +and target branch, then attempts one bounded POST. It transports caller-owned +strings, not commit inventories, bodies, code, or credentials. + +**This authorizes reconciliation of an ancestor range; it does not prove the +original push boundary.** The authenticated source run attests `push_after`/head, +source repository ID, default branch, and exact +`.github/workflows/team-memory-post-merge.yml` path. It cannot establish the +original `push_before` or push flags. The shared invocation action validates +`push_before` as an ancestor, `push_after` as an ancestor of the current source +tip, and retrieves the complete paginated range centrally (up to 1,000 commits, +100 PRs, and a three-minute discovery budget). It rejects truncated/duplicate +inventories, incomplete PR associations, changed identities/tips, and divergent +ranges. Requests explicitly mark `range_origin: authorized-reconciliation` and +require paged PR/source evidence and current-wiki comparison. Requests preserve source and coordinator identities separately, require current-source/wiki comparison, and authorize wiki knowledge writes only. @@ -145,21 +158,22 @@ partial, failed, needs-review, missing, or wrong-wiki receipts fail the job. No merged PRs is an explicit preflight skip before Azure login. There is one dispatch POST and one agent submission, without automatic retries. -An uploaded artifact does not prove accepted dispatch. On ambiguous dispatch, -timeout, or agent/write outcome, inspect coordinator runs and confirmed wiki +An accepted dispatch does not prove coordinator execution or admission to the +queue. On ambiguous dispatch, timeout, or agent/write outcome, inspect coordinator runs and confirmed wiki state before manually reconciling through this queue. It does not serialize chat/direct/external writers or prove a timed-out hosted invocation stopped. ## Offline checks With the pinned IssueLens action files available locally, set -`ISSUELENS_CLIENT_PATH` to their `issuelens_action.py` and run: +`ISSUELENS_ACTIONS_PATH` to their `.github/actions` directory and run: ```text python -m unittest discover -s .github/actions/queue-team-memory/tests ``` -The focused CI workflow loads the same immutable client and shared-action -revision. These checks pass the shared dispatcher's actual snapshot through -Java preflight and use fake API/stream responses; they make no dispatch, Azure, -or wiki calls. +The focused CI workflow fetches the same immutable shared actions for tests only. +Checks use actual action schemas and generic dispatch payloads, source-ID/read +authentication, `main`/`develop` handling, complete range/PR discovery, and final +receipts with fake API/stream responses. They make no live dispatch, Azure, or +wiki calls. diff --git a/.github/actions/queue-team-memory/team_memory.py b/.github/actions/queue-team-memory/team_memory.py deleted file mode 100644 index e7875094..00000000 --- a/.github/actions/queue-team-memory/team_memory.py +++ /dev/null @@ -1,267 +0,0 @@ -"""Java coordinator provenance adapter for the pinned IssueLens client.""" - -import argparse -import importlib.util -import json -import os -import re -import tempfile -import urllib.request -from datetime import datetime, timedelta, timezone -from pathlib import Path - -COORDINATOR = "microsoft/vscode-java-pack" -SOURCE_WORKFLOW = ".github/workflows/team-memory-post-merge.yml" -COORDINATOR_WORKFLOW = ".github/workflows/team-memory-coordinator.yml" -SOURCE_INPUTS = ("SOURCE_RUN_ID", "SOURCE_RUN_ATTEMPT", "SOURCE_ARTIFACT_ID") -MAX_SOURCE_BYTES = 64 * 1024 -# Reviewed public sources with an existing policy targeting the Java Pack wiki. -SOURCES = { - COORDINATOR: (104967329, "main"), - "microsoft/vscode-java-debug": (102584737, "main"), - "microsoft/vscode-java-test": (110522074, "main"), - "microsoft/vscode-java-dependency": (129053101, "main"), - "microsoft/vscode-maven": (116921700, "main"), - "microsoft/vscode-gradle": (216314492, "develop"), - "microsoft/java-debug": (102583752, "main"), - "microsoft/build-server-for-gradle": (626812412, "develop"), -} - - -def require(condition, message): - if not condition: - raise ValueError(message) - - -def positive(value): - require(re.fullmatch(r"[1-9][0-9]{0,14}", str(value)), "Invalid positive numeric identifier") - return int(value) - - -def full_sha(value): - require(isinstance(value, str) and re.fullmatch(r"[0-9a-f]{40}", value) - and value != "0" * 40, "Expected a full nonzero commit SHA") - return value - - -def unique_object(pairs): - result = {} - for key, value in pairs: - require(key not in result, "Duplicate JSON key") - result[key] = value - return result - - -def read_json(path, limit): - with path.open("rb") as source: - content = source.read(limit + 1) - require(len(content) <= limit, "Source JSON exceeds its size limit") - result = json.loads(content, object_pairs_hook=unique_object) - require(isinstance(result, dict), "Expected a JSON object") - return result - - -def matching_metadata(actual, expected): - return (isinstance(actual, dict) and actual == expected - and all(type(actual[key]) is type(value) for key, value in expected.items())) - - -class NoRedirect(urllib.request.HTTPRedirectHandler): - def redirect_request(self, *args, **kwargs): - return None - - -def github_read(path, token): - require(isinstance(token, str) and token.strip(), "Required GitHub token is empty") - request = urllib.request.Request( - "https://api.github.com" + path, - headers={"Authorization": "Bearer " + token, "Accept": "application/vnd.github+json", - "Content-Type": "application/json", "X-GitHub-Api-Version": "2022-11-28"}, - ) - with urllib.request.build_opener(NoRedirect()).open(request, timeout=30) as response: - content = response.read(4 * 1024 * 1024 + 1) - require(len(content) <= 4 * 1024 * 1024, "GitHub response exceeds its size limit") - result = json.loads(content, object_pairs_hook=unique_object) - require(isinstance(result, dict), "Invalid GitHub response") - return result - - -def project(repository, token): - require(repository in SOURCES, "Source repository is not allowlisted") - identity, branch = SOURCES[repository] - result = github_read(f"/repos/{repository}", token) - require(result.get("full_name") == repository and result.get("id") == identity - and result.get("default_branch") == branch - and result.get("private") is False and result.get("archived") is False, - "Repository identity, public visibility, or default branch changed; review the allowlist") - return result - - -def workflow_context(repository, workflow, event_name, token): - require(os.environ["GITHUB_REPOSITORY"] == repository - and os.environ["GITHUB_EVENT_NAME"] == event_name, "Unexpected workflow context") - event = read_json(Path(os.environ["GITHUB_EVENT_PATH"]), 32 * 1024 * 1024) - current = project(repository, token) - require(isinstance(event.get("repository"), dict) - and event["repository"].get("full_name") == repository - and event["repository"].get("id") == current["id"], "Event repository mismatch") - reference = "refs/heads/" + current["default_branch"] - require(os.environ["GITHUB_REF"] == reference - and os.environ["GITHUB_WORKFLOW_REF"] == repository + "/" + workflow + "@" + reference, - "Run the trusted workflow from the current default branch") - full_sha(os.environ["GITHUB_WORKFLOW_SHA"]) - return current, event - - -def source_event_path(): - return Path(os.environ["RUNNER_TEMP"]) / "issuelens-team-memory-source" / "source-event.json" - - -def outputs(values): - with open(os.environ["GITHUB_OUTPUT"], "a", encoding="utf-8") as output: - for name, value in values.items(): - output.write(f"{name}={value}\n") - - -def selection(): - repository = os.environ.get("SOURCE_REPOSITORY", "") - require(repository in SOURCES, "Supply an allowlisted source_repository") - values = [os.environ.get(name, "") for name in SOURCE_INPUTS] - manual = os.environ.get("DISPATCH_PR", "") - require((all(values) and not manual) or (not any(values) and manual), - "Supply all three source identifiers or one manual PR, not both") - identifiers = tuple(positive(value) for value in values) if all(values) else None - if manual: - positive(manual) - workflow_context(COORDINATOR, COORDINATOR_WORKFLOW, "workflow_dispatch", os.environ["COORDINATOR_TOKEN"]) - return repository, identifiers - - -def select_source(): - repository, _ = selection() - outputs({"source-name": repository.split("/")[1], - "external": "true" if repository != COORDINATOR else "false"}) - - -def verify_source(repository, current, identifiers): - run_id, attempt, artifact_id = identifiers - token = os.environ["GH_TOKEN"] - run = github_read(f"/repos/{repository}/actions/runs/{run_id}/attempts/{attempt}", token) - require(type(run.get("id")) is int and run["id"] == run_id - and type(run.get("run_attempt")) is int and run["run_attempt"] == attempt and run.get("event") == "push" - and run.get("path") == SOURCE_WORKFLOW and run.get("head_branch") == current["default_branch"] - and isinstance(run.get("repository"), dict) and run["repository"].get("id") == current["id"] - and run["repository"].get("full_name") == repository - and isinstance(run.get("head_repository"), dict) and run["head_repository"].get("id") == current["id"] - and run["head_repository"].get("full_name") == repository, - "Source run is not the allowlisted default-branch push workflow") - head = full_sha(run.get("head_sha")) - actors = [run.get(name) for name in ("actor", "triggering_actor")] - require(all(isinstance(actor, dict) and isinstance(actor.get("login"), str) - and re.fullmatch(r"[A-Za-z0-9-]+(?:\[bot\])?", actor["login"]) for actor in actors), - "Invalid source run actors") - artifact = github_read(f"/repos/{repository}/actions/artifacts/{artifact_id}", token) - origin = artifact.get("workflow_run") - require(artifact.get("id") == artifact_id and artifact.get("expired") is False - and artifact.get("name") == f"issuelens-team-memory-source-{attempt}" - and isinstance(artifact.get("digest"), str) - and re.fullmatch(r"sha256:[0-9a-f]{64}", artifact["digest"]) - and type(artifact.get("size_in_bytes")) is int and 0 < artifact["size_in_bytes"] <= MAX_SOURCE_BYTES - and isinstance(origin, dict) and origin.get("id") == run_id - and origin.get("repository_id") == current["id"] and origin.get("head_repository_id") == current["id"] - and origin.get("head_branch") == current["default_branch"] and origin.get("head_sha") == head, - "Source artifact is expired, oversized, lacks a digest, or belongs to a different run") - created = datetime.fromisoformat(artifact["created_at"].replace("Z", "+00:00")) - expires = datetime.fromisoformat(artifact["expires_at"].replace("Z", "+00:00")) - require(created.tzinfo is not None and expires.tzinfo is not None - and created < expires <= created + timedelta(days=7) - and created <= datetime.now(timezone.utc) < expires, "Source artifact is outside its seven-day lifetime") - return { - "repository": repository, "repository_id": current["id"], "base_ref": current["default_branch"], - "event_name": "push", "event_action": "push", - "actor_login": actors[0]["login"], "triggering_actor": actors[1]["login"], - "workflow_ref": repository + "/" + SOURCE_WORKFLOW + "@refs/heads/" + current["default_branch"], - "workflow_sha": head, "run_id": run_id, "run_attempt": attempt, - } - - -def validate_dispatch(): - repository, identifiers = selection() - current = project(repository, os.environ["GH_TOKEN"]) - if identifiers is not None: - verify_source(repository, current, identifiers) - outputs({"automatic": "true" if identifiers is not None else "false", "source-repository": repository, - **({"source-run-id": identifiers[0], "source-artifact-id": identifiers[2]} if identifiers else {})}) - print("Validated queued source metadata" if identifiers else "Validated manual source selection") - - -def load_client(): - spec = importlib.util.spec_from_file_location("issuelens_client", os.environ["ISSUELENS_CLIENT_PATH"]) - require(spec is not None and spec.loader is not None, "Pinned IssueLens client is unavailable") - client = importlib.util.module_from_spec(spec) - spec.loader.exec_module(client) - return client - - -def coordinator_metadata(): - return { - "coordinator_repository": COORDINATOR, "coordinator_workflow_ref": os.environ["GITHUB_WORKFLOW_REF"], - "coordinator_workflow_sha": full_sha(os.environ["GITHUB_WORKFLOW_SHA"]), - "coordinator_run_id": positive(os.environ["GITHUB_RUN_ID"]), - "coordinator_run_attempt": positive(os.environ["GITHUB_RUN_ATTEMPT"]), - "required_wiki_repository": COORDINATOR, - } - - -def preflight(): - repository, identifiers = selection() - current = project(repository, os.environ["GH_TOKEN"]) - central = coordinator_metadata() - client = load_client() - client.display_options() - if identifiers is not None: - metadata = verify_source(repository, current, identifiers) - snapshot = read_json(source_event_path(), MAX_SOURCE_BYTES) - require(set(snapshot) == {"metadata", "event"} and matching_metadata(snapshot["metadata"], metadata), - "Source artifact metadata does not match the verified run") - push = snapshot["event"] - require(isinstance(push, dict) - and set(push) == {"repository", "ref", "before", "after", "created", "deleted", - "forced", "commits", "head_commit"} - and push["repository"] == {"id": current["id"], "full_name": repository} - and isinstance(push["commits"], list) - and all(isinstance(item, dict) and set(item) == {"id"} for item in push["commits"]) - and push["head_commit"] == {"id": metadata["workflow_sha"]}, - "Invalid identity-only push artifact") - try: - envelope = client.prepare_push_memory(repository, current, push, {**metadata, **central}) - except client.SkippedRequest as skipped: - outputs({"eligible": "false", "status": "skipped", "skip-reason": str(skipped)}) - print(f"No maintenance request: {skipped}") - return - else: - merged = client.read_merged_pr(repository, current, positive(os.environ["DISPATCH_PR"])) - metadata = {"repository": repository, "repository_id": current["id"], "base_ref": current["default_branch"], - "event_name": "workflow_dispatch", "event_action": "workflow_dispatch", **merged, **central} - envelope = {"metadata": metadata, "request": client.build_team_memory_request(metadata)} - envelope["request_type"] = "team-memory" - descriptor, path = tempfile.mkstemp(prefix="issuelens-request-", suffix=".json", dir=os.environ["RUNNER_TEMP"]) - with os.fdopen(descriptor, "w", encoding="utf-8") as request: - json.dump(envelope, request) - outputs({"request-path": path, "eligible": "true"}) - print("Prepared bounded Java wiki maintenance request") - - -def run(command): - try: - {"select-source": select_source, "validate-dispatch": validate_dispatch, "preflight": preflight}[command]() - except ValueError as error: - raise SystemExit(f"::error::{error}") from None - except Exception: - raise SystemExit("::error::Java team-memory validation failed; no agent request was sent") from None - - -if __name__ == "__main__": - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("command", choices=("select-source", "validate-dispatch", "preflight")) - run(parser.parse_args().command) diff --git a/.github/actions/queue-team-memory/tests/test_team_memory.py b/.github/actions/queue-team-memory/tests/test_team_memory.py index af8563f9..8a74945f 100644 --- a/.github/actions/queue-team-memory/tests/test_team_memory.py +++ b/.github/actions/queue-team-memory/tests/test_team_memory.py @@ -3,23 +3,43 @@ import io import json import os -import re +import subprocess import tempfile import unittest -from datetime import datetime, timedelta, timezone from email.message import Message from pathlib import Path from unittest.mock import Mock, patch -ACTION = Path(__file__).resolve().parents[1] -ROOT = ACTION.parents[2] -spec = importlib.util.spec_from_file_location("java_team_memory", ACTION / "team_memory.py") -action = importlib.util.module_from_spec(spec) -spec.loader.exec_module(action) +import yaml + +ROOT = Path(__file__).resolve().parents[4] +SHARED = Path(os.environ["ISSUELENS_ACTIONS_PATH"]).resolve() +PIN = "a81d2d96167fc0e69ac631c2edc85f858e693289" +COORDINATOR = "microsoft/vscode-java-pack" + + +def load(path, name): + spec = importlib.util.spec_from_file_location(name, path) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +def read_yaml(path): + return yaml.load(path.read_text(encoding="utf-8"), Loader=yaml.BaseLoader) + + +invoker = load(SHARED / "issuelens" / "issuelens_action.py", "shared_invoker") +dispatcher = load(SHARED / "queue-team-memory" / "dispatch.py", "shared_dispatcher") +source_workflow = read_yaml(ROOT / ".github" / "workflows" / "team-memory-post-merge.yml") +central_workflow = read_yaml(ROOT / ".github" / "workflows" / "team-memory-coordinator.yml") +source_job = source_workflow["jobs"]["dispatch"] +central_job = central_workflow["jobs"]["reconcile"] +ALLOWED = json.loads(central_job["env"]["ISSUELENS_SOURCE_REPOSITORIES"]) class Response: - def __init__(self, content=b"", status=204): + def __init__(self, content=b"", status=200): self.content = io.BytesIO(content) self.status = status self.headers = Message() @@ -31,650 +51,497 @@ def __enter__(self): def __exit__(self, *args): return False + def read(self, limit): + return self.content.read(limit) + def readline(self, limit): return self.content.readline(limit) -class TeamMemoryTests(unittest.TestCase): - @classmethod - def setUpClass(cls): - cls.client = action.load_client() - +class ContractTests(unittest.TestCase): def setUp(self): - self.temporary = tempfile.TemporaryDirectory() - self.addCleanup(self.temporary.cleanup) - self.directory = Path(self.temporary.name) - self.repository = "microsoft/vscode-gradle" + self.temp = tempfile.TemporaryDirectory() + self.addCleanup(self.temp.cleanup) + self.directory = Path(self.temp.name) self.before, self.merge, self.after, self.tip = [letter * 40 for letter in "daef"] - self.project = self.make_project(self.repository) - self.central = self.make_project(action.COORDINATOR) - self.coordinator_workflow = {"id": 789, "path": action.COORDINATOR_WORKFLOW, "state": "active"} - self.coordinator_branch = {"name": "main", "commit": {"sha": self.tip}} + self.source = "microsoft/vscode-gradle" + self.central = self.project(COORDINATOR) + self.source_project = self.project(self.source) self.environment = { - "GITHUB_REPOSITORY": action.COORDINATOR, "GITHUB_EVENT_NAME": "workflow_dispatch", + "GITHUB_REPOSITORY": COORDINATOR, "GITHUB_EVENT_NAME": "workflow_dispatch", "GITHUB_REF": "refs/heads/main", "GITHUB_SHA": self.tip, "GITHUB_WORKFLOW_SHA": self.tip, - "GITHUB_WORKFLOW_REF": action.COORDINATOR + "/" + action.COORDINATOR_WORKFLOW + "@refs/heads/main", + "GITHUB_WORKFLOW_REF": COORDINATOR + "/.github/workflows/team-memory-coordinator.yml@refs/heads/main", "GITHUB_ACTOR": "maintainer", "GITHUB_TRIGGERING_ACTOR": "rerunner", "GITHUB_RUN_ID": "999", "GITHUB_RUN_ATTEMPT": "1", "GITHUB_EVENT_PATH": str(self.directory / "event.json"), "RUNNER_TEMP": str(self.directory), "GITHUB_OUTPUT": str(self.directory / "output.txt"), - "COORDINATOR_TOKEN": "test-coordinator-token", "GH_TOKEN": "test-source-token", - "DISPATCH_TOKEN": "test-dispatch-token", "SOURCE_REPOSITORY": self.repository, - "COORDINATOR_REPOSITORY": action.COORDINATOR, - "COORDINATOR_WORKFLOW": "team-memory-coordinator.yml", "COORDINATOR_REF": "main", - "SOURCE_RUN_ID": "123456", "SOURCE_RUN_ATTEMPT": "2", "SOURCE_ARTIFACT_ID": "456", - "DISPATCH_PR": "", "OUTPUT_MODE": "quiet", "SUMMARY_MODE": "none", + "GH_TOKEN": "test-coordinator-token", "SOURCE_GH_TOKEN": "test-source-token", + "SOURCE_REPOSITORY": self.source, "SOURCE_REPOSITORIES": json.dumps(ALLOWED), + "SOURCE_RUN_ID": "123456", "SOURCE_RUN_ATTEMPT": "2", + "PUSH_BEFORE": self.before, "PUSH_AFTER": self.after, "DISPATCH_PR": "", + "REQUEST_TYPE": "team-memory", "TASK_INPUT": "", "OUTPUT_MODE": "quiet", "SUMMARY_MODE": "none", "AGENT_URL": "https://test.services.ai.azure.com/agents/test/protocols/invocations", - "AGENT_SCOPE": "api://test/.default", + "AGENT_SCOPE": "api://test/.default", "DISPATCH_TOKEN": "test-dispatch-token", + "COORDINATOR_REPOSITORY": COORDINATOR, "COORDINATOR_WORKFLOW": "team-memory-coordinator.yml", + "COORDINATOR_REF": "main", } self.env_patch = patch.dict(os.environ, self.environment, clear=True) self.env_patch.start() self.addCleanup(self.env_patch.stop) - self.write_event({"repository": self.central}) + Path(os.environ["GITHUB_EVENT_PATH"]).write_text(json.dumps({"repository": self.central})) self.source_run = { - "id": 123456, "run_attempt": 2, "event": "push", "path": action.SOURCE_WORKFLOW, - "head_branch": "develop", "head_sha": self.after, - "repository": self.project, "head_repository": self.project, - "actor": {"login": "maintainer"}, "triggering_actor": {"login": "rerunner"}, - } - now = datetime.now(timezone.utc) - timedelta(minutes=1) - self.artifact = { - "id": 456, "name": "issuelens-team-memory-source-2", "expired": False, "size_in_bytes": 2048, - "digest": "sha256:" + "1" * 64, "created_at": now.isoformat(), - "expires_at": (now + timedelta(days=7)).isoformat(), - "workflow_run": { - "id": 123456, "repository_id": self.project["id"], "head_repository_id": self.project["id"], - "head_branch": "develop", "head_sha": self.after, - }, + "id": 123456, "run_attempt": 2, "event": "push", "path": invoker.DISPATCH_WORKFLOW, + "head_branch": "develop", "head_sha": self.after, "repository": self.source_project, + "head_repository": self.source_project, "actor": {"login": "maintainer"}, + "triggering_actor": {"login": "rerunner"}, } - self.metadata = { - "repository": self.repository, "repository_id": self.project["id"], "base_ref": "develop", - "event_name": "push", "event_action": "push", "actor_login": "maintainer", "triggering_actor": "rerunner", - "workflow_ref": self.repository + "/" + action.SOURCE_WORKFLOW + "@refs/heads/develop", - "workflow_sha": self.after, "run_id": 123456, "run_attempt": 2, - } - self.push = { - "repository": {"id": self.project["id"], "full_name": self.repository}, "ref": "refs/heads/develop", - "before": self.before, "after": self.after, "created": False, "deleted": False, "forced": False, - "commits": [{"id": self.merge}, {"id": self.after}], "head_commit": {"id": self.after}, - } - self.snapshot = {"metadata": self.metadata, "event": self.push} + self.branch = {"name": "develop", "commit": {"sha": self.tip}} + self.ancestry = self.comparison(self.after, 1) + self.inventory = self.comparison(self.before, 2, [self.merge, self.after]) self.pull = { "number": 27, "merged": True, "state": "closed", "merge_commit_sha": self.merge, - "merged_at": "2026-10-08T00:00:00Z", "base": {"ref": "develop", "repo": self.project}, - } - self.comparison = { - "base_commit": {"sha": self.before}, "merge_base_commit": {"sha": self.before}, "status": "ahead", - "ahead_by": 2, "behind_by": 0, "total_commits": 2, "commits": [{"sha": self.after}], + "merged_at": "2026-10-09T00:00:00Z", "base": {"ref": "develop", "repo": self.source_project}, } - node = { - "number": 27, "state": "MERGED", "merged": True, "mergedAt": self.pull["merged_at"], - "baseRefName": "develop", "baseRepository": {"databaseId": self.project["id"], "nameWithOwner": self.repository}, - "mergeCommit": {"oid": self.merge}, - } - connection = {"totalCount": 1, "pageInfo": {"hasNextPage": False}, "nodes": [node]} - self.graphql = {"data": {"repository": { - "databaseId": self.project["id"], "nameWithOwner": self.repository, - "defaultBranchRef": {"name": "develop", "target": {"oid": self.tip}}, - "c0": {"oid": self.merge, "associatedPullRequests": copy.deepcopy(connection)}, - "c1": {"oid": self.after, "associatedPullRequests": copy.deepcopy(connection)}, - }}} - self.api_reads = [] - self.mock_read = patch.object(action, "github_read", side_effect=self.api_read).start() - self.addCleanup(patch.stopall) - patch.object(self.client, "github_read", side_effect=lambda path, payload=None, **kwargs: self.api_read( - path, kwargs.get("token") or os.environ["GH_TOKEN"], payload)).start() - patch.object(action, "load_client", return_value=self.client).start() + self.graphql = self.associations([self.merge, self.after]) + self.target_workflow = {"id": 789, "path": ".github/workflows/team-memory-coordinator.yml", "state": "active"} + self.target_branch = {"name": "main", "commit": {"sha": self.tip}} + self.submit_response = Response() + self.requests = [] self.opener = Mock() - self.opener.open.return_value = Response() - patch("urllib.request.build_opener", return_value=self.opener).start() - self.azure = patch.object(self.client.subprocess, "check_output", return_value="test-azure-token").start() - self.log = patch("sys.stdout", new_callable=io.StringIO).start() - - def make_project(self, repository): - identity, branch = action.SOURCES[repository] - return {"full_name": repository, "id": identity, "default_branch": branch, "private": False, "archived": False} - - def api_read(self, path, token, payload=None): - self.api_reads.append((path, token, payload)) + self.opener.open.side_effect = self.respond + self.patches = [ + patch("urllib.request.build_opener", return_value=self.opener), + patch.object(invoker.subprocess, "check_output", return_value="test-azure-token"), + patch("sys.stdout", new_callable=io.StringIO), + ] + _, self.azure, self.log = [item.start() for item in self.patches] + for item in self.patches: + self.addCleanup(item.stop) + + def project(self, name): + branch = "develop" if name in {"microsoft/vscode-gradle", "microsoft/build-server-for-gradle"} else "main" + return {"id": ALLOWED[name], "full_name": name, "default_branch": branch, "visibility": "public"} + + def comparison(self, before, count, shas=()): + return {"base_commit": {"sha": before}, "merge_base_commit": {"sha": before}, + "status": "ahead", "ahead_by": count, "behind_by": 0, "total_commits": count, + "commits": [{"sha": sha} for sha in shas]} + + def associations(self, shas): + result = {"databaseId": self.source_project["id"], "nameWithOwner": self.source, + "defaultBranchRef": {"name": self.source_project["default_branch"], "target": {"oid": self.tip}}} + for index, sha in enumerate(shas): + node = {"number": 27, "state": "MERGED", "merged": True, "mergedAt": self.pull["merged_at"], + "baseRefName": self.source_project["default_branch"], + "baseRepository": {"databaseId": self.source_project["id"], "nameWithOwner": self.source}, + "mergeCommit": {"oid": self.merge}} + result[f"c{index}"] = {"oid": sha, "associatedPullRequests": { + "totalCount": 1, "pageInfo": {"hasNextPage": False}, "nodes": [node]}} + return {"data": {"repository": result}} + + def select_source(self, name): + self.source = name + self.source_project = self.project(name) + os.environ["SOURCE_REPOSITORY"] = name + self.source_run.update(repository=self.source_project, head_repository=self.source_project, + head_branch=self.source_project["default_branch"]) + self.branch["name"] = self.source_project["default_branch"] + self.pull["base"] = {"ref": self.source_project["default_branch"], "repo": self.source_project} + self.graphql = self.associations([self.merge, self.after]) + + def respond(self, request, timeout): + self.requests.append(request) + path = request.full_url.removeprefix("https://api.github.com") + if request.full_url == os.environ["AGENT_URL"]: + return self.submit_response + if path.endswith("/dispatches"): + return Response(status=204) responses = { - f"/repos/{action.COORDINATOR}": self.central, f"/repos/{self.repository}": self.project, - f"/repos/{self.repository}/actions/runs/123456/attempts/2": self.source_run, - f"/repos/{self.repository}/actions/artifacts/456": self.artifact, - f"/repos/{self.repository}/pulls/27": self.pull, - f"/repos/{self.repository}/compare/{self.before}...{self.after}?per_page=1&page=2": self.comparison, - f"/repos/{action.COORDINATOR}/actions/workflows/team-memory-coordinator.yml": self.coordinator_workflow, - f"/repos/{action.COORDINATOR}/branches/main": self.coordinator_branch, - "/graphql": self.graphql, + f"/repos/{COORDINATOR}": self.central, f"/repos/{self.source}": self.source_project, + f"/repos/{COORDINATOR}/actions/workflows/team-memory-coordinator.yml": self.target_workflow, + f"/repos/{COORDINATOR}/branches/main": self.target_branch, + f"/repos/{self.source}/actions/runs/123456/attempts/2": self.source_run, + f"/repos/{self.source}/branches/{self.source_project['default_branch']}": self.branch, + f"/repos/{self.source}/compare/{self.after}...{self.tip}?per_page=1&page=2": self.ancestry, + f"/repos/{self.source}/compare/{self.before}...{self.after}?per_page=100&page=1": self.inventory, + f"/repos/{self.source}/pulls/27": self.pull, "/graphql": self.graphql, } - self.assertIn(path, responses, "Unexpected API access") - return copy.deepcopy(responses[path]) - - def write_event(self, event): - Path(os.environ["GITHUB_EVENT_PATH"]).write_text(json.dumps(event), encoding="utf-8") - - def write_source(self, content=None): - path = action.source_event_path() - path.parent.mkdir(exist_ok=True) - path.write_bytes(json.dumps(self.snapshot).encode() if content is None else content) - return path + self.assertIn(path, responses, "Unexpected network path") + return Response(json.dumps(responses[path]).encode()) - def output(self): + def outputs(self): path = Path(os.environ["GITHUB_OUTPUT"]) return dict(line.split("=", 1) for line in path.read_text().splitlines()) if path.exists() else {} - def envelope(self): - return json.loads(Path(self.output()["request-path"]).read_text()) - - def select_push(self): - os.environ.update( - GITHUB_REPOSITORY=self.repository, GITHUB_EVENT_NAME="push", GITHUB_SHA=self.after, - GITHUB_WORKFLOW_SHA=self.after, GITHUB_REF="refs/heads/develop", - GITHUB_WORKFLOW_REF=self.metadata["workflow_ref"], GITHUB_RUN_ID="123456", GITHUB_RUN_ATTEMPT="2", - ) - self.write_event(self.push) - - def execute(self, command): - if command in {"prepare-dispatch", "dispatch"}: - self.client.run(command) - else: - action.run(command) + def preflight(self): + invoker.run("preflight") self.azure.assert_not_called() - - def test_identity_only_source_excludes_all_untrusted_text_and_tokens(self): - self.select_push() - event = copy.deepcopy(self.push) - event["commits"][0]["message"] = "UNTRUSTED_COMMIT_MESSAGE" - event["repository"]["description"] = "UNTRUSTED_REPOSITORY_TEXT" - self.write_event(event) - self.execute("prepare-dispatch") - path = Path(self.output()["source-event-path"]) - self.assertEqual(json.loads(path.read_bytes()), self.snapshot) - self.assertNotIn("UNTRUSTED", path.read_text()) - self.assertNotIn("test-source-token", path.read_text()) - self.opener.open.assert_not_called() - - def test_shared_dispatcher_snapshot_passes_java_preflight_without_translation(self): - coordinator_environment = dict(os.environ) - self.select_push() - self.execute("prepare-dispatch") - self.assertEqual(json.loads(action.source_event_path().read_bytes()), self.snapshot) - Path(os.environ["GITHUB_OUTPUT"]).unlink() - os.environ.clear() - os.environ.update(coordinator_environment) - self.write_event({"repository": self.central}) - self.execute("validate-dispatch") - Path(os.environ["GITHUB_OUTPUT"]).unlink() - self.execute("preflight") - metadata = self.envelope()["metadata"] - self.assertEqual(metadata["repository"], self.repository) - self.assertEqual(metadata["push_after"], self.after) - self.assertEqual(metadata["required_wiki_repository"], action.COORDINATOR) - self.opener.open.assert_not_called() - - def test_maximum_inventory_fits_and_matches_upstream_limits(self): - self.assertEqual(self.client.MAX_PUSH_COMMITS, 1000) - self.assertEqual(action.MAX_SOURCE_BYTES, self.client.MAX_SOURCE_BYTES) - self.select_push() - event = copy.deepcopy(self.push) - event["commits"] = [{"id": f"{number:040x}"} for number in range(1, 1000)] + [{"id": self.after}] - self.write_event(event) - self.execute("prepare-dispatch") - self.assertLess(action.source_event_path().stat().st_size, action.MAX_SOURCE_BYTES) - - def test_every_allowlisted_source_uses_its_own_identity_and_default_branch(self): - original = dict(os.environ) - for repository, (identity, branch) in action.SOURCES.items(): - with self.subTest(repository=repository), patch.dict(os.environ, { - **original, "GITHUB_REPOSITORY": repository, "GITHUB_EVENT_NAME": "push", - "GITHUB_REF": "refs/heads/" + branch, "GITHUB_SHA": self.after, "GITHUB_WORKFLOW_SHA": self.after, - "GITHUB_WORKFLOW_REF": repository + "/" + action.SOURCE_WORKFLOW + "@refs/heads/" + branch, - }, clear=True), patch.object(self.client, "github_read", return_value=self.make_project(repository)): - self.write_event({**self.push, "repository": {"id": identity, "full_name": repository}, - "ref": "refs/heads/" + branch}) - self.client.prepare_dispatch() - snapshot = json.loads(action.source_event_path().read_bytes()) - self.assertEqual(snapshot["metadata"]["repository_id"], identity) - self.assertEqual(snapshot["metadata"]["base_ref"], branch) - self.assertEqual(snapshot["event"]["repository"]["full_name"], repository) - - def test_push_rejects_unsafe_or_incomplete_identities(self): - self.select_push() - cases = [ - {"forced": True}, {"created": True}, {"deleted": True}, {"before": "0" * 40}, - {"after": self.before}, {"head_commit": {"id": self.merge}}, {"ref": "refs/heads/main"}, - {"commits": []}, {"commits": [{"id": self.after}] * 2}, - {"commits": [{"id": self.after}] * 1001}, {"commits": [{"id": "short"}]}, - ] - for changes in cases: - with self.subTest(changes=changes): - self.write_event({**self.push, **changes}) - with self.assertRaises(SystemExit): - self.execute("prepare-dispatch") - self.assertFalse(action.source_event_path().exists()) - self.assertEqual(self.output(), {}) - - def test_push_rejects_wrong_workflow_branch_revision_and_event(self): - self.select_push() - original = dict(os.environ) - for name, value in ( - ("GITHUB_WORKFLOW_REF", self.repository + "/.github/workflows/untrusted.yml@refs/heads/develop"), - ("GITHUB_REF", "refs/heads/main"), ("GITHUB_WORKFLOW_SHA", self.tip), - ("GITHUB_EVENT_NAME", "pull_request"), - ): - with self.subTest(name=name), patch.dict(os.environ, {**original, name: value}, clear=True): - with self.assertRaises(SystemExit): - self.execute("prepare-dispatch") - self.assertEqual(self.output(), {}) - - def test_dispatch_sends_one_post_to_central_main_with_separate_tokens(self): - self.select_push() - self.write_source() - self.execute("dispatch") - self.assertEqual([(path, token) for path, token, _ in self.api_reads], [ - (f"/repos/{self.repository}", "test-source-token"), - (f"/repos/{action.COORDINATOR}", "test-dispatch-token"), - (f"/repos/{action.COORDINATOR}/actions/workflows/team-memory-coordinator.yml", "test-dispatch-token"), - (f"/repos/{action.COORDINATOR}/branches/main", "test-dispatch-token"), - ]) - self.opener.open.assert_called_once() - request = self.opener.open.call_args.args[0] - self.assertEqual(request.full_url, - f"https://api.github.com/repos/{action.COORDINATOR}/actions/workflows/team-memory-coordinator.yml/dispatches") - self.assertEqual(request.get_method(), "POST") - self.assertEqual(json.loads(request.data), {"ref": "main", "inputs": { - "source_repository": self.repository, "source_run_id": "123456", - "source_run_attempt": "2", "source_artifact_id": "456", - }}) - self.assertNotIn(b"test-", request.data) - self.assertIn("completion is reported by the coordinator", self.log.getvalue()) - - def test_dispatch_does_not_retry_ambiguous_failure_or_leak_transport_details(self): - self.select_push() - self.write_source() - self.opener.open.side_effect = OSError("PRIVATE_TRANSPORT_DETAIL") + self.assertFalse(any(request.get_method() == "POST" and not request.full_url.endswith("/graphql") + for request in self.requests)) + return json.loads(Path(self.outputs()["request-path"]).read_text()) if self.outputs().get("eligible") == "true" else None + + def dispatch_inputs(self): + content = source_job["steps"][0]["with"]["workflow-inputs"] + replacements = {"github.repository": self.source, "github.run_id": "123456", "github.run_attempt": "2", + "github.event.before": self.before, "github.sha": self.after} + for name, value in replacements.items(): + content = content.replace("${{ " + name + " }}", value) + return content + + def test_source_main_and_develop_dispatch_scalar_inputs_to_central_main(self): + for name in (COORDINATOR, "microsoft/vscode-gradle"): + with self.subTest(source=name): + self.select_source(name) + self.requests.clear() + os.environ["WORKFLOW_INPUTS"] = self.dispatch_inputs() + dispatcher.run() + posted, = [request for request in self.requests if request.get_method() == "POST"] + self.assertEqual(json.loads(posted.data), {"ref": "main", "inputs": { + "source_repository": name, "source_run_id": "123456", "source_run_attempt": "2", + "push_before": self.before, "push_after": self.after, + }}) + self.assertEqual(len(self.requests), 4) + self.assertTrue(all(request.get_header("Authorization") == "Bearer test-dispatch-token" + for request in self.requests)) + self.azure.assert_not_called() + + def test_generic_dispatch_payload_drives_shared_invocation_without_artifacts(self): + os.environ["WORKFLOW_INPUTS"] = self.dispatch_inputs() + dispatcher.run() + inputs = json.loads(self.requests[-1].data)["inputs"] + os.environ.update({name.upper(): value for name, value in inputs.items()}) + self.requests.clear() + metadata = self.preflight()["metadata"] + self.assertEqual(metadata["repository"], self.source) + self.assertEqual(metadata["required_wiki_repository"], COORDINATOR) + self.assertEqual(metadata["range_origin"], "authorized-reconciliation") + self.assertFalse(any("artifact" in request.full_url for request in self.requests)) + + def test_dispatch_ambiguous_post_has_no_retry_or_completion_claim(self): + os.environ["WORKFLOW_INPUTS"] = self.dispatch_inputs() + original = self.respond + + def fail_post(request, timeout): + if request.get_method() == "POST": + self.requests.append(request) + raise OSError("PRIVATE_TRANSPORT_DETAIL") + return original(request, timeout) + + self.opener.open.side_effect = fail_post with self.assertRaisesRegex(SystemExit, "outcome is unknown"): - self.execute("dispatch") - self.opener.open.assert_called_once() + dispatcher.run() + self.assertEqual(sum(request.get_method() == "POST" for request in self.requests), 1) self.assertNotIn("PRIVATE_TRANSPORT_DETAIL", self.log.getvalue()) - self.assertEqual(self.output(), {}) - - def test_explicit_empty_dispatch_token_fails_before_any_network_access(self): - self.select_push() - for token in ("", " \t"): - with self.subTest(token=token), patch.dict(os.environ, {"DISPATCH_TOKEN": token}): - with self.assertRaisesRegex(SystemExit, "dispatch-token must be non-empty"): - self.execute("dispatch") - self.mock_read.assert_not_called() + self.assertEqual(self.outputs(), {}) + + def test_dispatch_empty_token_and_invalid_payloads_fail_before_network(self): + for changes in ({"DISPATCH_TOKEN": ""}, {"WORKFLOW_INPUTS": '{"run":1}'}, + {"WORKFLOW_INPUTS": '{"run":"1","run":"2"}'}, + {"WORKFLOW_INPUTS": "\u20ac" * 22000}, {"COORDINATOR_WORKFLOW": "../bad.yml"}): + with self.subTest(changes=changes), patch.dict(os.environ, changes), self.assertRaises(SystemExit): + dispatcher.run() self.opener.open.assert_not_called() - def test_dispatch_revalidates_saved_push_inventory_before_target_access(self): - self.select_push() - self.snapshot["event"]["before"] = self.merge - self.write_source() - with self.assertRaisesRegex(SystemExit, "Push commit inventory"): - self.execute("dispatch") - self.opener.open.assert_not_called() - self.assertFalse(any(token == "test-dispatch-token" for _, token, _ in self.api_reads)) - - def test_shared_dispatch_rejects_inactive_or_wrong_target_workflow_without_post(self): - self.select_push() - self.write_source() - original = copy.deepcopy(self.coordinator_workflow) - for changes in ({"state": "disabled_manually"}, {"path": ".github/workflows/untrusted.yml"}): - with self.subTest(changes=changes): - self.coordinator_workflow = {**original, **changes} - with self.assertRaisesRegex(SystemExit, "Coordinator workflow identity"): - self.execute("dispatch") - self.opener.open.assert_not_called() - - def test_shared_dispatch_rejects_wrong_target_branch_without_post(self): - self.select_push() - self.write_source() - self.coordinator_branch["name"] = "develop" - with self.assertRaisesRegex(SystemExit, "Coordinator branch identity"): - self.execute("dispatch") - self.opener.open.assert_not_called() + def test_inactive_target_workflow_or_wrong_branch_cannot_dispatch(self): + os.environ["WORKFLOW_INPUTS"] = self.dispatch_inputs() + self.target_workflow["state"] = "disabled_manually" + with self.assertRaisesRegex(SystemExit, "workflow is inactive"): + dispatcher.run() + self.target_workflow["state"] = "active" + self.target_branch["name"] = "develop" + with self.assertRaisesRegex(SystemExit, "branch identity"): + dispatcher.run() + self.assertFalse(any(request.get_method() == "POST" for request in self.requests)) + + def test_all_eight_trusted_ids_use_current_source_default_branch(self): + for name, identity in ALLOWED.items(): + with self.subTest(source=name): + self.select_source(name) + metadata = self.preflight()["metadata"] + self.assertEqual(metadata["repository_id"], identity) + self.assertEqual(metadata["base_ref"], self.source_project["default_branch"]) + self.assertEqual(metadata["coordinator_repository"], COORDINATOR) + self.assertEqual(metadata["required_wiki_repository"], COORDINATOR) + + def test_source_and_coordinator_identity_and_read_credentials_stay_separate(self): + envelope = self.preflight() + metadata = envelope["metadata"] + self.assertEqual(metadata["workflow_sha"], self.after) + self.assertEqual(metadata["coordinator_workflow_sha"], self.tip) + self.assertEqual(metadata["run_id"], 123456) + self.assertEqual(metadata["coordinator_run_id"], 999) + self.assertEqual(self.requests[0].get_header("Authorization"), "Bearer test-coordinator-token") + self.assertTrue(all(request.get_header("Authorization") == "Bearer test-source-token" + for request in self.requests[1:])) + self.assertNotIn("test-source-token", json.dumps(envelope)) + self.assertIn("This never overrides repository policy", envelope["request"]["input"]) + self.assertIn("Do not modify source or issues", envelope["request"]["input"]) - def test_invalid_mixed_and_missing_inputs_fail_before_network(self): - cases = [ - {"SOURCE_REPOSITORY": ""}, {"SOURCE_REPOSITORY": "microsoft/IssueLens"}, - {"SOURCE_RUN_ID": ""}, {"SOURCE_RUN_ATTEMPT": ""}, {"SOURCE_ARTIFACT_ID": ""}, - {"SOURCE_RUN_ID": "1; unsafe"}, {"SOURCE_RUN_ATTEMPT": "0"}, {"SOURCE_ARTIFACT_ID": "01"}, - {"DISPATCH_PR": "27"}, {name: "" for name in action.SOURCE_INPUTS}, - ] - for changes in cases: - with self.subTest(changes=changes), patch.dict(os.environ, changes): - with self.assertRaises(SystemExit): - self.execute("select-source") - self.mock_read.assert_not_called() - - def test_only_central_default_branch_coordinator_may_select_a_source(self): - for change in ( - {"GITHUB_REPOSITORY": self.repository}, {"GITHUB_EVENT_NAME": "push"}, - {"GITHUB_REF": "refs/heads/feature"}, {"GITHUB_WORKFLOW_SHA": "short"}, - {"GITHUB_WORKFLOW_REF": action.COORDINATOR + "/.github/workflows/untrusted.yml@refs/heads/main"}, - ): - with self.subTest(change=change), patch.dict(os.environ, change): - with self.assertRaises(SystemExit): - self.execute("select-source") - self.assertEqual(self.output(), {}) - - def test_source_selection_precedes_external_token_minting(self): - self.execute("select-source") - self.assertEqual(self.output(), {"external": "true", "source-name": "vscode-gradle"}) - self.assertEqual(self.api_reads, [(f"/repos/{action.COORDINATOR}", "test-coordinator-token", None)]) - - def test_java_pack_manual_source_needs_no_external_credentials(self): - self.repository = action.COORDINATOR - self.project = self.central - self.pull["base"] = {"ref": "main", "repo": self.central} - os.environ.update({**{name: "" for name in action.SOURCE_INPUTS}, - "DISPATCH_PR": "27", "SOURCE_REPOSITORY": action.COORDINATOR, - "GH_TOKEN": "test-coordinator-token"}) - self.execute("select-source") - self.assertEqual(self.output(), {"external": "false", "source-name": "vscode-java-pack"}) - Path(os.environ["GITHUB_OUTPUT"]).unlink() - self.execute("preflight") - metadata = self.envelope()["metadata"] - self.assertEqual(metadata["repository"], action.COORDINATOR) - self.assertEqual(metadata["base_ref"], "main") - self.assertEqual(metadata["required_wiki_repository"], action.COORDINATOR) - self.assertTrue(all(token == "test-coordinator-token" for _, token, _ in self.api_reads)) - - def test_validates_external_source_run_and_artifact_before_download(self): - self.execute("validate-dispatch") - self.assertEqual(self.output(), { - "automatic": "true", "source-repository": self.repository, - "source-run-id": "123456", "source-artifact-id": "456", - }) - self.assertEqual([path for path, _, _ in self.api_reads], [ - f"/repos/{action.COORDINATOR}", f"/repos/{self.repository}", - f"/repos/{self.repository}/actions/runs/123456/attempts/2", - f"/repos/{self.repository}/actions/artifacts/456", - ]) - self.assertTrue(all(token == "test-source-token" for _, token, _ in self.api_reads[1:])) + def test_requested_before_is_an_ancestor_selection_not_original_push_proof(self): + requested = "c" * 40 + os.environ["PUSH_BEFORE"] = requested + self.before = requested + self.inventory = self.comparison(requested, 2, [self.merge, self.after]) + envelope = self.preflight() + self.assertEqual(envelope["metadata"]["push_before"], requested) + self.assertIn("not an attestation of the original push boundary", envelope["request"]["input"]) + self.assertEqual(set(invoker.SOURCE_INPUTS), {"SOURCE_RUN_ID", "SOURCE_RUN_ATTEMPT", "PUSH_BEFORE", "PUSH_AFTER"}) + + def test_unknown_source_mixed_inputs_and_empty_source_token_fail_before_network(self): + for changes in ({"SOURCE_REPOSITORY": "microsoft/IssueLens"}, {"SOURCE_RUN_ID": ""}, + {"SOURCE_RUN_ATTEMPT": "0"}, {"PUSH_BEFORE": ""}, {"PUSH_AFTER": "short"}, + {"PUSH_BEFORE": self.after}, {"DISPATCH_PR": "27"}, + {"SOURCE_GH_TOKEN": ""}, {"SOURCE_GH_TOKEN": "bad\ncredential"}): + with self.subTest(changes=changes), patch.dict(os.environ, changes), self.assertRaises(SystemExit): + self.preflight() self.opener.open.assert_not_called() - def test_rejects_changed_identity_visibility_or_default_branch(self): - original = copy.deepcopy(self.project) - for changes in ( - {"id": original["id"] + 1}, {"full_name": "microsoft/other"}, {"default_branch": "main"}, - {"private": True}, {"private": None}, {"archived": True}, - ): + def test_changed_source_id_canonical_name_or_private_visibility_is_rejected(self): + original = copy.deepcopy(self.source_project) + for changes in ({"id": original["id"] + 1}, {"full_name": "microsoft/renamed"}, + {"visibility": "private"}, {"visibility": "internal"}): with self.subTest(changes=changes): - self.project = {**original, **changes} + self.source_project = {**original, **changes} with self.assertRaises(SystemExit): - self.execute("validate-dispatch") - self.assertFalse(any("/actions/" in path for path, _, _ in self.api_reads)) + self.preflight() + self.assertFalse(any("/actions/runs/" in request.full_url for request in self.requests)) - def test_forged_run_cannot_proceed_to_artifact_validation(self): + def test_forged_source_run_attempt_workflow_head_or_branch_cannot_invoke(self): original = copy.deepcopy(self.source_run) - for changes in ( - {"id": 123457}, {"run_attempt": 1}, {"event": "workflow_dispatch"}, {"path": "untrusted.yml"}, - {"head_branch": "main"}, {"head_sha": "short"}, - {"repository": self.central}, {"head_repository": self.central}, - {"actor": {"login": "unsafe\nactor"}}, - ): + for changes in ({"id": 123457}, {"run_attempt": 1}, {"event": "pull_request"}, + {"path": ".github/workflows/untrusted.yml"}, {"head_sha": self.tip}, + {"head_branch": "main"}, {"head_repository": self.central}): with self.subTest(changes=changes): self.source_run = {**original, **changes} - with self.assertRaises(SystemExit): - self.execute("validate-dispatch") - self.assertFalse(any("/artifacts/" in path for path, _, _ in self.api_reads)) - - def test_foreign_expired_oversized_or_digestless_artifacts_fail(self): - original = copy.deepcopy(self.artifact) - for changes in ( - {"id": 457}, {"name": "issuelens-team-memory-source-1"}, {"expired": True}, {"digest": None}, - {"digest": "short"}, {"size_in_bytes": 0}, {"size_in_bytes": True}, - {"size_in_bytes": action.MAX_SOURCE_BYTES + 1}, - {"workflow_run": {**original["workflow_run"], "id": 123457}}, - {"workflow_run": {**original["workflow_run"], "repository_id": self.central["id"]}}, - {"workflow_run": {**original["workflow_run"], "head_sha": self.merge}}, - ): - with self.subTest(changes=changes): - self.artifact = {**original, **changes} - with self.assertRaises(SystemExit): - self.execute("validate-dispatch") - self.assertEqual(self.output(), {}) - - def test_expiry_timestamps_are_checked_not_only_expired_flag(self): - original = copy.deepcopy(self.artifact) - now = datetime.now(timezone.utc) - for changes in ( - {"expires_at": (now - timedelta(seconds=1)).isoformat()}, - {"expires_at": (now + timedelta(days=8)).isoformat()}, - {"created_at": (now + timedelta(hours=1)).isoformat()}, - {"expires_at": "2026-10-01T00:00:00"}, {"created_at": "invalid"}, - ): + with self.assertRaisesRegex(SystemExit, "Source run does not match"): + self.preflight() + self.assertEqual(self.outputs(), {}) + + def test_wrong_coordinator_workflow_or_nondefault_ref_is_rejected(self): + for changes in ({"GITHUB_REF": "refs/heads/feature"}, { + "GITHUB_WORKFLOW_REF": COORDINATOR + "/.github/workflows/untrusted.yml@refs/heads/main"}): + with self.subTest(changes=changes), patch.dict(os.environ, changes), self.assertRaises(SystemExit): + self.preflight() + self.assertEqual(self.outputs(), {}) + + def test_divergent_head_or_current_source_tip_race_is_rejected(self): + self.ancestry["behind_by"] = 1 + with self.assertRaisesRegex(SystemExit, "complete fast-forward"): + self.preflight() + self.ancestry["behind_by"] = 0 + self.graphql["data"]["repository"]["defaultBranchRef"]["target"]["oid"] = self.merge + with self.assertRaisesRegex(SystemExit, "changed during reconciliation"): + self.preflight() + + def test_truncated_duplicate_divergent_or_missing_head_range_fails(self): + original = copy.deepcopy(self.inventory) + for changes in ({"commits": [{"sha": self.after}]}, + {"commits": [{"sha": self.after}, {"sha": self.after}]}, + {"commits": [{"sha": self.merge}, {"sha": self.tip}]}, + {"status": "diverged"}, {"total_commits": 1001}, {"total_commits": 0}): with self.subTest(changes=changes): - self.artifact = {**original, **changes} + self.inventory = {**original, **changes} with self.assertRaises(SystemExit): - self.execute("validate-dispatch") - - def test_preflight_reuses_discovery_and_keeps_source_and_coordinator_separate(self): - self.write_source() - self.execute("preflight") - envelope = self.envelope() - metadata = envelope["metadata"] - self.assertEqual(metadata["repository"], self.repository) - self.assertEqual(metadata["base_ref"], "develop") - self.assertEqual(metadata["workflow_sha"], self.after) - self.assertEqual(metadata["coordinator_repository"], action.COORDINATOR) - self.assertEqual(metadata["coordinator_workflow_sha"], self.tip) - self.assertEqual(metadata["required_wiki_repository"], action.COORDINATOR) - self.assertEqual(metadata["run_id"], 123456) - self.assertEqual(metadata["coordinator_run_id"], 999) - self.assertEqual(metadata["source_tip_sha"], self.tip) - self.assertEqual([item["pull_number"] for item in metadata["pull_requests"]], [27]) - self.assertEqual(envelope["request_type"], "team-memory") - self.assertIn("This never overrides repository policy", envelope["request"]["input"]) - self.assertIn("Do not modify source or issues", envelope["request"]["input"]) - self.assertIn("current wiki knowledge", envelope["request"]["input"]) - self.assertTrue(any("/actions/artifacts/" in path for path, _, _ in self.api_reads)) - - def test_malformed_or_forged_downloaded_snapshot_is_rejected(self): - original = copy.deepcopy(self.snapshot) - cases = [ - {**original, "secret": "forbidden"}, - {**original, "metadata": {**original["metadata"], "run_id": 999}}, - {**original, "metadata": {**original["metadata"], "run_attempt": 2.0}}, - {**original, "metadata": {**original["metadata"], "repository": action.COORDINATOR}}, - {**original, "event": {**original["event"], "message": "forbidden"}}, - {**original, "event": {**original["event"], "commits": [{"id": self.after, "message": "forbidden"}]}}, - {**original, "event": {**original["event"], "repository": {"id": self.central["id"], "full_name": action.COORDINATOR}}}, - {**original, "event": {**original["event"], "after": self.merge}}, - ] - for snapshot in cases: - with self.subTest(snapshot=snapshot): - self.write_source(json.dumps(snapshot).encode()) - with self.assertRaises(SystemExit): - self.execute("preflight") - for content in (b"{}", b'{"metadata":{},"metadata":{},"event":{}}', b"x" * (action.MAX_SOURCE_BYTES + 1)): - with self.subTest(content=content[:50]): - self.write_source(content) - with self.assertRaises(SystemExit): - self.execute("preflight") - self.assertEqual(self.output(), {}) - - def test_discovery_rejects_incomplete_range_and_association_pagination(self): - self.write_source() - self.comparison["total_commits"] = 3 - with self.assertRaisesRegex(SystemExit, "complete fast-forward"): - self.execute("preflight") - self.comparison["total_commits"] = 2 + self.preflight() + self.assertEqual(self.outputs(), {}) + + def test_complete_1000_commit_pagination_and_partial_final_page(self): + for count in (102, 1000): + with self.subTest(count=count): + shas = [f"{number:040x}" for number in range(1, count)] + [self.after] + pages = [self.comparison(self.before, count, shas[start:start + 100]) for start in range(0, count, 100)] + with patch.object(invoker, "github_read", side_effect=pages) as read: + actual = invoker.read_reconciliation_inventory(self.source, self.before, self.after, float("inf")) + self.assertEqual(actual, shas) + self.assertEqual(read.call_count, len(pages)) + self.assertTrue(all(call.kwargs["token"] == "test-source-token" for call in read.call_args_list)) + self.assertTrue(read.call_args.args[0].endswith(f"page={len(pages)}")) + + def test_missing_second_range_page_and_incomplete_pr_associations_fail(self): + shas = [f"{number:040x}" for number in range(1, 102)] + [self.after] + pages = [self.comparison(self.before, len(shas), shas[:100]), + self.comparison(self.before, len(shas), shas[100:-1])] + with patch.object(invoker, "github_read", side_effect=pages), self.assertRaisesRegex(ValueError, "pagination"): + invoker.read_reconciliation_inventory(self.source, self.before, self.after, float("inf")) self.graphql["data"]["repository"]["c0"]["associatedPullRequests"]["pageInfo"]["hasNextPage"] = True with self.assertRaisesRegex(SystemExit, "associations are incomplete"): - self.execute("preflight") - self.assertEqual(self.output(), {}) - - def test_rebased_merge_uses_pinned_rest_revalidation(self): - self.write_source() - for key in ("c0", "c1"): - self.graphql["data"]["repository"][key]["associatedPullRequests"]["nodes"][0]["mergeCommit"] = None - self.execute("preflight") - self.assertEqual(self.envelope()["metadata"]["pull_requests"][0]["merge_commit_sha"], self.merge) - self.assertEqual(sum(path.endswith("/pulls/27") for path, _, _ in self.api_reads), 1) - - def test_no_merged_pr_is_an_explicit_skip_before_login(self): - self.write_source() - for key in ("c0", "c1"): - self.graphql["data"]["repository"][key]["associatedPullRequests"] = { - "totalCount": 0, "pageInfo": {"hasNextPage": False}, "nodes": [], - } - self.execute("preflight") - self.assertEqual(self.output(), {"eligible": "false", "status": "skipped", "skip-reason": "no_merged_pull_requests"}) - - def test_manual_external_pr_uses_source_default_branch_and_same_wiki(self): - os.environ.update({**{name: "" for name in action.SOURCE_INPUTS}, "DISPATCH_PR": "27"}) - self.execute("validate-dispatch") - self.assertEqual(self.output(), {"automatic": "false", "source-repository": self.repository}) + self.preflight() + + def test_rebase_merge_rest_validation_and_no_merged_pr_skip(self): + for name in ("c0", "c1"): + self.graphql["data"]["repository"][name]["associatedPullRequests"]["nodes"][0]["mergeCommit"] = None + self.assertEqual(self.preflight()["metadata"]["pull_requests"][0]["merge_commit_sha"], self.merge) + self.assertEqual(sum(request.full_url.endswith("/pulls/27") for request in self.requests), 1) + for name in ("c0", "c1"): + self.graphql["data"]["repository"][name]["associatedPullRequests"] = { + "totalCount": 0, "pageInfo": {"hasNextPage": False}, "nodes": []} Path(os.environ["GITHUB_OUTPUT"]).unlink() - self.execute("preflight") - metadata = self.envelope()["metadata"] - self.assertEqual(metadata["repository"], self.repository) + self.assertIsNone(self.preflight()) + self.assertEqual(self.outputs(), {"eligible": "false", "status": "skipped", "skip-reason": "no_merged_pull_requests"}) + + def test_manual_pr_uses_same_shared_invoker_and_source_default_branch(self): + os.environ.update({**{name: "" for name in invoker.SOURCE_INPUTS}, "DISPATCH_PR": "27"}) + metadata = self.preflight()["metadata"] self.assertEqual(metadata["base_ref"], "develop") self.assertEqual(metadata["pull_number"], 27) - self.assertEqual(metadata["required_wiki_repository"], action.COORDINATOR) - self.assertNotIn("workflow_sha", metadata) - self.assertFalse(any("/actions/" in path for path, _, _ in self.api_reads)) - - def test_manual_unmerged_or_wrong_source_pr_is_rejected(self): - os.environ.update({**{name: "" for name in action.SOURCE_INPUTS}, "DISPATCH_PR": "27"}) - original = copy.deepcopy(self.pull) - for changes in ( - {"merged": False}, {"state": "open"}, {"merge_commit_sha": "short"}, - {"base": {"ref": "main", "repo": self.project}}, - {"base": {"ref": "develop", "repo": self.central}}, - ): - with self.subTest(changes=changes): - self.pull = {**original, **changes} - with self.assertRaises(SystemExit): - self.execute("preflight") - self.assertEqual(self.output(), {}) - - def result(self, status="no-change"): - return { - "source_repository": self.repository, "push_before": self.before, "push_after": self.after, - "status": status, "wiki_repository": action.COORDINATOR, "wiki_sha": "b" * 40, - "reason": "Verified source and wiki", "results": [ - {"pull_number": 27, "merge_commit_sha": self.merge, "status": status, "reason": "Verified"}, - ], - } + self.assertEqual(metadata["required_wiki_repository"], COORDINATOR) + self.assertFalse(any("/actions/runs/" in request.full_url for request in self.requests)) + self.pull["merged"] = False + with self.assertRaisesRegex(SystemExit, "not merged"): + self.preflight() + + def result(self): + return {"source_repository": self.source, "push_before": self.before, "push_after": self.after, + "status": "no-change", "wiki_repository": COORDINATOR, "wiki_sha": "b" * 40, + "reason": "Verified source and wiki", "results": [ + {"pull_number": 27, "merge_commit_sha": self.merge, "status": "no-change", "reason": "Verified"}]} def submit(self, result, done=True): - os.environ["REQUEST_PATH"] = self.output()["request-path"] - stream = 'data: ' + json.dumps({"type": "assistant.message", "data": {"content": json.dumps(result)}}) + "\n\n" + os.environ["REQUEST_PATH"] = self.outputs()["request-path"] + stream = "data: " + json.dumps({"type": "assistant.message", "data": {"content": json.dumps(result)}}) + "\n\n" if done: stream += 'event: done\ndata: {"invocation_id":"test","session_id":"test"}\n\n' - self.opener.open.return_value = Response(stream.encode()) - self.client.run("submit") + self.submit_response = Response(stream.encode()) + invoker.run("submit") - def test_completed_stream_with_valid_receipt_is_required_for_success(self): - self.write_source() - self.execute("preflight") + def test_completed_stream_requires_valid_receipt_with_shared_wiki_and_sha(self): + self.preflight() self.submit(self.result()) - self.assertEqual(self.output()["status"], "no-change") - self.assertEqual(self.output()["wiki-repository"], action.COORDINATOR) - self.opener.open.assert_called_once() + self.assertEqual(self.outputs()["status"], "no-change") + self.assertEqual(self.outputs()["wiki-repository"], COORDINATOR) + self.assertEqual(sum(request.full_url == os.environ["AGENT_URL"] for request in self.requests), 1) + self.assertEqual(self.opener.open.call_args.kwargs["timeout"], 300) - def test_stream_completion_alone_wrong_wiki_missing_pr_or_receipt_is_not_success(self): - self.write_source() - self.execute("preflight") + def test_wrong_wiki_source_missing_pr_or_success_shaped_status_cannot_succeed(self): + self.preflight() original = self.result() - for changes in ( - {"wiki_repository": "microsoft/IssueLens"}, {"wiki_sha": None}, {"results": []}, - {"source_repository": action.COORDINATOR}, {"push_after": self.merge}, {"status": "completed"}, - ): - with self.subTest(changes=changes): - with self.assertRaises(SystemExit): - self.submit({**original, **changes}) - self.assertNotIn("status", self.output()) - - def test_incomplete_stream_fails_without_retry(self): - self.write_source() - self.execute("preflight") - with self.assertRaisesRegex(SystemExit, "completion event"): + for changes in ({"wiki_repository": "microsoft/IssueLens"}, {"wiki_sha": None}, + {"source_repository": COORDINATOR}, {"results": []}, {"status": "completed"}): + with self.subTest(changes=changes), self.assertRaises(SystemExit): + self.submit({**original, **changes}) + self.assertNotIn("status", self.outputs()) + + def test_incomplete_stream_is_unknown_and_never_retried(self): + self.preflight() + with self.assertRaisesRegex(SystemExit, "completion event; outcome unknown"): self.submit(self.result(), done=False) - self.opener.open.assert_called_once() - self.assertNotIn("status", self.output()) - - def test_failed_or_partial_batch_does_not_claim_whole_batch_success(self): - self.write_source() - self.execute("preflight") - with self.assertRaisesRegex(SystemExit, "batch incomplete"): - self.submit(self.result("failed")) - self.assertEqual(self.output()["status"], "failed") - self.opener.open.assert_called_once() + self.assertEqual(sum(request.full_url == os.environ["AGENT_URL"] for request in self.requests), 1) + self.assertNotIn("status", self.outputs()) - def test_partial_publication_is_reported_but_fails_the_coordinator(self): - self.write_source() + def test_partial_publication_keeps_receipt_but_fails_batch(self): connection = self.graphql["data"]["repository"]["c1"]["associatedPullRequests"] - connection["nodes"].append({**copy.deepcopy(connection["nodes"][0]), - "number": 28, "mergeCommit": {"oid": self.after}}) + connection["nodes"].append({**copy.deepcopy(connection["nodes"][0]), "number": 28, + "mergeCommit": {"oid": self.after}}) connection["totalCount"] = 2 - self.execute("preflight") + self.preflight() result = self.result() result["status"] = "partial" result["results"].append({"pull_number": 28, "merge_commit_sha": self.after, "status": "needs-review", "reason": "Incomplete evidence"}) with self.assertRaisesRegex(SystemExit, "batch incomplete"): self.submit(result) - self.assertEqual(self.output()["status"], "partial") - self.assertEqual(self.output()["wiki-repository"], action.COORDINATOR) - self.opener.open.assert_called_once() + self.assertEqual(self.outputs()["status"], "partial") + self.assertEqual(self.outputs()["wiki-repository"], COORDINATOR) class WiringTests(unittest.TestCase): - def test_only_verified_sources_are_allowlisted(self): - self.assertEqual(len(action.SOURCES), 8) - self.assertNotIn("microsoft/vscode-spring-initializr", action.SOURCES) - self.assertNotIn("microsoft/vscode-spring-boot-dashboard", action.SOURCES) - self.assertEqual(action.SOURCES["microsoft/build-server-for-gradle"][1], "develop") - - def test_workflows_share_new_opt_in_and_only_coordinator_invokes(self): - source = (ROOT / ".github/workflows/team-memory-post-merge.yml").read_text() - central = (ROOT / ".github/workflows/team-memory-coordinator.yml").read_text() - shared = Path(os.environ["ISSUELENS_CLIENT_PATH"]).resolve().parent.parent / "queue-team-memory" / "action.yml" - composite = shared.read_text() - self.assertFalse((ACTION / "action.yml").exists()) - for command in ("prepare_dispatch", "dispatch", "push_snapshot"): - self.assertFalse(hasattr(action, command), "Source dispatch belongs only to the shared action") - for workflow in (source, central): - self.assertIn("ISSUELENS_TEAM_MEMORY_COORDINATOR_ENABLED == 'true'", workflow) - self.assertNotIn("ISSUELENS_TEAM_MEMORY_ENABLED", workflow) - self.assertIn("permissions: {}", workflow) - self.assertNotIn("actions/checkout@", source) - self.assertNotIn("python", source) - self.assertIn("persist-credentials: false", central) - self.assertNotIn("concurrency:", source) - self.assertNotIn("workflow_dispatch:", source) - for text in (source, composite): - self.assertNotIn("azure/login", text) - self.assertNotIn("agent-url", text) - self.assertNotIn("id-token:", text) - self.assertIn("group: issuelens-team-memory-wiki-microsoft-vscode-java-pack", central) - self.assertIn("queue: max\n cancel-in-progress: false", central) - self.assertIn("digest-mismatch: error", central) - self.assertNotIn("steps.source-token.outputs.token || github.token", central) - self.assertIn("repository: ${{ steps.source.outputs.source-repository }}", central) - self.assertLess(central.index("validate-dispatch"), central.index("actions/download-artifact@")) - self.assertLess(central.index("team_memory.py preflight"), central.index("azure/login@")) - self.assertIn('python3 -I "$ISSUELENS_CLIENT_PATH" submit', central) - self.assertIn("retention-days: 7", composite) - self.assertIn("GH_TOKEN: ${{ inputs.source-token }}", composite) - self.assertIn("DISPATCH_TOKEN: ${{ inputs.dispatch-token }}", composite) - for name, value in (("repository", action.COORDINATOR), ("workflow", "team-memory-coordinator.yml"), ("ref", "main")): - self.assertIn(f"coordinator-{name}: {value}", source) - - def test_client_pin_matches_coordinator_tests_and_documentation(self): - source = (ROOT / ".github/workflows/team-memory-post-merge.yml").read_text() - match = re.search(r"uses: microsoft/IssueLens/\.github/actions/queue-team-memory@([0-9a-f]{40})", source) - self.assertIsNotNone(match) - revision = match.group(1) - for relative in (".github/workflows/team-memory-coordinator.yml", ".github/workflows/team-memory-tests.yml"): - self.assertIn("ref: " + revision, (ROOT / relative).read_text()) - self.assertIn(revision, (ACTION / "README.md").read_text()) + def test_workflows_use_latest_real_input_schemas_and_only_shared_invocation(self): + queue = read_yaml(SHARED / "queue-team-memory" / "action.yml") + invoke = read_yaml(SHARED / "issuelens" / "action.yml") + dispatch, = source_job["steps"] + self.assertEqual(dispatch["uses"], "microsoft/IssueLens/.github/actions/queue-team-memory@" + PIN) + self.assertTrue(set(dispatch["with"]).issubset(queue["inputs"])) + selection, auth, invocation = central_job["steps"] + self.assertEqual(invocation["uses"], "microsoft/IssueLens/.github/actions/issuelens@" + PIN) + self.assertTrue(set(invocation["with"]).issubset(invoke["inputs"])) + self.assertEqual(invocation["with"]["source-repositories"], "${{ env.ISSUELENS_SOURCE_REPOSITORIES }}") + for name in ("source_repository", "source_run_id", "source_run_attempt", "push_before", "push_after"): + self.assertEqual(invocation["with"][name.replace("_", "-")], "${{ inputs." + name + " }}") + self.assertNotIn("continue-on-error", invocation) + self.assertNotIn("run", invocation) + preflight, login, submit = invoke["runs"]["steps"] + self.assertEqual(preflight["env"]["SOURCE_GH_TOKEN"], "${{ inputs.source-github-token }}") + self.assertTrue(all(step["if"] == "steps.preflight.outputs.eligible == 'true'" for step in (login, submit))) + self.assertEqual(len(queue["runs"]["steps"]), 1) + self.assertEqual(queue["runs"]["steps"][0]["run"], 'python3 -I "$GITHUB_ACTION_PATH/dispatch.py"') + + def test_source_gates_and_one_central_queue_cover_manual_and_automatic_requests(self): + self.assertEqual(source_workflow["on"], {"push": {"branches": ["main"]}}) + self.assertEqual(set(central_workflow["on"]), {"workflow_dispatch"}) + self.assertEqual(central_workflow["concurrency"], { + "group": "issuelens-team-memory-wiki-microsoft-vscode-java-pack", "queue": "max", "cancel-in-progress": "false"}) + self.assertNotIn("concurrency", source_workflow) + for workflow, job in ((source_workflow, source_job), (central_workflow, central_job)): + self.assertEqual(workflow["permissions"], {}) + self.assertIn("ISSUELENS_TEAM_MEMORY_COORDINATOR_ENABLED == 'true'", job["if"]) + self.assertIn("github.event.repository.default_branch", job["if"]) + for flag in ("created", "deleted", "forced"): + self.assertIn(f"github.event.{flag} == false", source_job["if"]) + self.assertIn("github.workflow_sha == github.sha", source_job["if"]) + self.assertEqual(source_job["permissions"], {"contents": "read", "actions": "write"}) + self.assertEqual(central_job["permissions"], { + "contents": "read", "actions": "read", "pull-requests": "read", "id-token": "write"}) + self.assertEqual(central_job["timeout-minutes"], "30") + inputs = central_workflow["on"]["workflow_dispatch"]["inputs"] + self.assertEqual(set(inputs), {"source_repository", "source_run_id", "source_run_attempt", + "push_before", "push_after", "pull_request_number"}) + self.assertIn("not an attested original push boundary", inputs["push_before"]["description"]) + + def test_trusted_allowlist_selection_precedes_single_source_read_authentication(self): + self.assertEqual(ALLOWED, { + COORDINATOR: 104967329, "microsoft/vscode-java-debug": 102584737, + "microsoft/vscode-java-test": 110522074, "microsoft/vscode-java-dependency": 129053101, + "microsoft/vscode-maven": 116921700, "microsoft/vscode-gradle": 216314492, + "microsoft/java-debug": 102583752, "microsoft/build-server-for-gradle": 626812412}) + selection, auth, invocation = central_job["steps"] + self.assertIn("'has($repository)'", selection["run"]) + self.assertIn('exit 1', selection["run"]) + self.assertNotIn("${{", selection["run"]) + self.assertEqual(auth["if"], "steps.selection.outputs.external == 'true'") + self.assertEqual(auth["with"]["repositories"], "${{ steps.selection.outputs.source-name }}") + self.assertEqual(auth["with"]["owner"], "microsoft") + self.assertEqual({key: value for key, value in auth["with"].items() if key.startswith("permission-")}, + {"permission-actions": "read", "permission-contents": "read", "permission-pull-requests": "read"}) + self.assertEqual(invocation["with"]["source-github-token"], + "${{ steps.selection.outputs.external == 'false' && github.token || steps.source-token.outputs.token }}") + self.assertEqual(invocation["with"]["output-mode"], "activity") + self.assertEqual(invocation["with"]["summary-mode"], "status") + + @unittest.skipIf(os.name == "nt", "Receiver selection runs on Ubuntu") + def test_actual_source_selection_script_restricts_scoped_app_authentication(self): + script = central_job["steps"][0]["run"] + with tempfile.TemporaryDirectory() as directory: + output = Path(directory) / "output.txt" + for repository in (*ALLOWED, "", "microsoft/IssueLens", "microsoft/vscode-spring-initializr", + "fork/vscode-java-pack", "microsoft/vscode-java-pack\ninjected=true"): + with self.subTest(source=repository): + output.unlink(missing_ok=True) + result = subprocess.run( + ["bash", "-e", "-c", script], capture_output=True, text=True, check=False, + env={**os.environ, "ISSUELENS_SOURCE_REPOSITORIES": json.dumps(ALLOWED), + "SOURCE_REPOSITORY": repository, "GITHUB_OUTPUT": str(output)}, + ) + if repository in ALLOWED: + self.assertEqual(result.returncode, 0, result.stderr) + external = "false" if repository == COORDINATOR else "true" + self.assertEqual(output.read_text().splitlines(), [ + f"source-name={repository.removeprefix('microsoft/')}", f"external={external}"]) + else: + self.assertEqual(result.returncode, 1) + self.assertIn("::error::Source repository is not allowlisted", result.stdout) + self.assertFalse(output.exists()) + + def test_no_runtime_artifacts_local_invoker_or_client_checkout(self): + for name in ("team-memory-post-merge.yml", "team-memory-coordinator.yml"): + content = (ROOT / ".github" / "workflows" / name).read_text() + for forbidden in ("artifact", "actions/checkout", "issuelens_action.py", "team_memory.py", + "ISSUELENS_CLIENT_PATH", "azure/login@"): + self.assertNotIn(forbidden, content) + self.assertFalse((Path(__file__).parents[1] / "team_memory.py").exists()) + self.assertFalse((Path(__file__).parents[1] / "action.yml").exists()) + tests = read_yaml(ROOT / ".github" / "workflows" / "team-memory-tests.yml") + self.assertEqual(tests["jobs"]["test"]["steps"][1]["with"]["ref"], PIN) + self.assertIn(PIN, (Path(__file__).parents[1] / "README.md").read_text()) if __name__ == "__main__": diff --git a/.github/workflows/team-memory-coordinator.yml b/.github/workflows/team-memory-coordinator.yml index be602309..32be7bcc 100644 --- a/.github/workflows/team-memory-coordinator.yml +++ b/.github/workflows/team-memory-coordinator.yml @@ -16,8 +16,12 @@ on: description: Source workflow attempt (required with source_run_id) required: false type: string - source_artifact_id: - description: Immutable source event artifact ID (required with source_run_id) + push_before: + description: Requested reconciliation ancestor SHA, not an attested original push boundary + required: false + type: string + push_after: + description: Source push run head SHA (required with source_run_id) required: false type: string pull_request_number: @@ -47,38 +51,32 @@ jobs: actions: read id-token: write env: - COORDINATOR_TOKEN: ${{ github.token }} - SOURCE_REPOSITORY: ${{ inputs.source_repository }} - SOURCE_RUN_ID: ${{ inputs.source_run_id }} - SOURCE_RUN_ATTEMPT: ${{ inputs.source_run_attempt }} - SOURCE_ARTIFACT_ID: ${{ inputs.source_artifact_id }} - DISPATCH_PR: ${{ inputs.pull_request_number }} - ISSUELENS_CLIENT_PATH: .issuelens-client/.github/actions/issuelens/issuelens_action.py - OUTPUT_MODE: activity - SUMMARY_MODE: status + ISSUELENS_SOURCE_REPOSITORIES: >- + {"microsoft/vscode-java-pack":104967329, + "microsoft/vscode-java-debug":102584737, + "microsoft/vscode-java-test":110522074, + "microsoft/vscode-java-dependency":129053101, + "microsoft/vscode-maven":116921700, + "microsoft/vscode-gradle":216314492, + "microsoft/java-debug":102583752, + "microsoft/build-server-for-gradle":626812412} steps: - - name: Load Java adapter from trusted workflow revision - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - ref: ${{ github.workflow_sha }} - persist-credentials: false - sparse-checkout: /.github/actions/queue-team-memory/ - sparse-checkout-cone-mode: false - - - name: Load reviewed IssueLens client - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - repository: microsoft/IssueLens - ref: 521b1ff80a70140e286ced7500d0b150cf9e6d6c # shared dispatcher prerequisite - path: .issuelens-client - persist-credentials: false - sparse-checkout: /.github/actions/issuelens/ - sparse-checkout-cone-mode: false - - - name: Validate coordinator and source selection + - name: Select allowlisted source for read-only authentication id: selection shell: bash - run: python3 -I .github/actions/queue-team-memory/team_memory.py select-source + env: + SOURCE_REPOSITORY: ${{ inputs.source_repository }} + run: | + if ! jq -e --arg repository "$SOURCE_REPOSITORY" 'has($repository)' <<< "$ISSUELENS_SOURCE_REPOSITORIES" > /dev/null; then + echo "::error::Source repository is not allowlisted" + exit 1 + fi + echo "source-name=${SOURCE_REPOSITORY#microsoft/}" >> "$GITHUB_OUTPUT" + if [[ "$SOURCE_REPOSITORY" == "microsoft/vscode-java-pack" ]]; then + echo "external=false" >> "$GITHUB_OUTPUT" + else + echo "external=true" >> "$GITHUB_OUTPUT" + fi - name: Get read-only token for the selected external source if: steps.selection.outputs.external == 'true' @@ -93,45 +91,23 @@ jobs: permission-contents: read permission-pull-requests: read - - name: Verify queued source before download - id: source - shell: bash - env: - GH_TOKEN: ${{ steps.selection.outputs.external == 'false' && github.token || steps.source-token.outputs.token }} - run: python3 -I .github/actions/queue-team-memory/team_memory.py validate-dispatch - - - name: Download verified source event - if: steps.source.outputs.automatic == 'true' - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + - name: Reconcile team memory and validate final receipts + uses: microsoft/IssueLens/.github/actions/issuelens@a81d2d96167fc0e69ac631c2edc85f858e693289 with: - artifact-ids: ${{ steps.source.outputs.source-artifact-id }} - run-id: ${{ steps.source.outputs.source-run-id }} - repository: ${{ steps.source.outputs.source-repository }} - github-token: ${{ steps.selection.outputs.external == 'false' && github.token || steps.source-token.outputs.token }} - path: ${{ runner.temp }}/issuelens-team-memory-source - digest-mismatch: error - - - name: Revalidate source and prepare bounded maintenance request - id: preflight - shell: bash - env: - GH_TOKEN: ${{ steps.selection.outputs.external == 'false' && github.token || steps.source-token.outputs.token }} - run: python3 -I .github/actions/queue-team-memory/team_memory.py preflight - - - name: Azure login for the existing agent endpoint - if: steps.preflight.outputs.eligible == 'true' - uses: azure/login@7ddb5af1ef8758cf1353cf3b42f940aee27ba21c # v3.0.2 - with: - client-id: ${{ secrets.AZURE_CLIENT_ID }} - tenant-id: ${{ secrets.AZURE_TENANT_ID }} - subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }} - - - name: Invoke IssueLens and validate the final maintenance receipt - if: steps.preflight.outputs.eligible == 'true' - shell: bash - env: - GH_TOKEN: ${{ steps.selection.outputs.external == 'false' && github.token || steps.source-token.outputs.token }} - REQUEST_PATH: ${{ steps.preflight.outputs.request-path }} - AGENT_URL: ${{ secrets.ISSUELENS_AGENT_URL }} - AGENT_SCOPE: ${{ secrets.ISSUELENS_AGENT_SCOPE }} - run: python3 -I "$ISSUELENS_CLIENT_PATH" submit + request-type: team-memory + github-token: ${{ github.token }} + source-repository: ${{ inputs.source_repository }} + source-repositories: ${{ env.ISSUELENS_SOURCE_REPOSITORIES }} + source-github-token: ${{ steps.selection.outputs.external == 'false' && github.token || steps.source-token.outputs.token }} + source-run-id: ${{ inputs.source_run_id }} + source-run-attempt: ${{ inputs.source_run_attempt }} + push-before: ${{ inputs.push_before }} + push-after: ${{ inputs.push_after }} + pull-request-number: ${{ inputs.pull_request_number }} + azure-client-id: ${{ secrets.AZURE_CLIENT_ID }} + azure-tenant-id: ${{ secrets.AZURE_TENANT_ID }} + azure-subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }} + agent-url: ${{ secrets.ISSUELENS_AGENT_URL }} + agent-scope: ${{ secrets.ISSUELENS_AGENT_SCOPE }} + output-mode: activity + summary-mode: status diff --git a/.github/workflows/team-memory-post-merge.yml b/.github/workflows/team-memory-post-merge.yml index 065f4ef1..aecc276d 100644 --- a/.github/workflows/team-memory-post-merge.yml +++ b/.github/workflows/team-memory-post-merge.yml @@ -12,18 +12,27 @@ jobs: ${{ vars.ISSUELENS_TEAM_MEMORY_COORDINATOR_ENABLED == 'true' && github.repository == 'microsoft/vscode-java-pack' && github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && - github.event_name == 'push' }} + github.event_name == 'push' && + github.workflow_sha == github.sha && + github.event.created == false && + github.event.deleted == false && + github.event.forced == false }} runs-on: ubuntu-latest - timeout-minutes: 5 + timeout-minutes: 10 permissions: contents: read actions: write steps: - name: Queue team-memory update - uses: microsoft/IssueLens/.github/actions/queue-team-memory@521b1ff80a70140e286ced7500d0b150cf9e6d6c + uses: microsoft/IssueLens/.github/actions/queue-team-memory@a81d2d96167fc0e69ac631c2edc85f858e693289 with: coordinator-repository: microsoft/vscode-java-pack coordinator-workflow: team-memory-coordinator.yml coordinator-ref: main - source-token: ${{ github.token }} dispatch-token: ${{ github.token }} + workflow-inputs: >- + {"source_repository":"${{ github.repository }}", + "source_run_id":"${{ github.run_id }}", + "source_run_attempt":"${{ github.run_attempt }}", + "push_before":"${{ github.event.before }}", + "push_after":"${{ github.sha }}"} diff --git a/.github/workflows/team-memory-tests.yml b/.github/workflows/team-memory-tests.yml index 0959b2ef..9e486301 100644 --- a/.github/workflows/team-memory-tests.yml +++ b/.github/workflows/team-memory-tests.yml @@ -1,4 +1,4 @@ -name: Java team-memory adapter checks +name: Java team-memory contract checks on: pull_request: @@ -23,11 +23,11 @@ jobs: with: persist-credentials: false - - name: Load reviewed IssueLens client and shared dispatcher + - name: Load pinned shared actions for offline contract tests only uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: repository: microsoft/IssueLens - ref: 521b1ff80a70140e286ced7500d0b150cf9e6d6c + ref: a81d2d96167fc0e69ac631c2edc85f858e693289 path: .issuelens-client persist-credentials: false sparse-checkout: | @@ -35,7 +35,7 @@ jobs: /.github/actions/queue-team-memory/ sparse-checkout-cone-mode: false - - name: Check source trust, dispatch, discovery, and receipts offline + - name: Check dispatch, source authorization, discovery, and receipts offline env: - ISSUELENS_CLIENT_PATH: .issuelens-client/.github/actions/issuelens/issuelens_action.py + ISSUELENS_ACTIONS_PATH: .issuelens-client/.github/actions run: python3 -m unittest discover -s .github/actions/queue-team-memory/tests From 69833dd5eaf953cb1e300cf2f1cce1f1b1943477 Mon Sep 17 00:00:00 2001 From: Changyong Gong Date: Sat, 10 Oct 2026 13:09:13 +0800 Subject: [PATCH 4/5] Keep team-memory tests in IssueLens Remove Java Pack's duplicate contract suite and dedicated test workflow. Retain shared-action runtime consumers and document IssueLens test ownership. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- .github/actions/queue-team-memory/README.md | 21 +- .../tests/test_team_memory.py | 548 ------------------ .github/workflows/team-memory-tests.yml | 41 -- 3 files changed, 6 insertions(+), 604 deletions(-) delete mode 100644 .github/actions/queue-team-memory/tests/test_team_memory.py delete mode 100644 .github/workflows/team-memory-tests.yml diff --git a/.github/actions/queue-team-memory/README.md b/.github/actions/queue-team-memory/README.md index 6341c37e..11980991 100644 --- a/.github/actions/queue-team-memory/README.md +++ b/.github/actions/queue-team-memory/README.md @@ -11,8 +11,8 @@ There is no unmerged prerequisite or hosted runtime deployment for action reuse. Manual PR reconciliation also runs through this central queue. There is no source preparation script, checkout, artifact upload/download, or local -invoker/provenance adapter. This directory retains setup guidance and offline -contract tests only. Shared actions own transport, source validation, discovery, +invoker/provenance adapter. This directory retains setup guidance only. +Shared actions own transport, source validation, discovery, Azure OIDC login, invocation, and final receipt checks. Both workflows require **`ISSUELENS_TEAM_MEMORY_COORDINATOR_ENABLED=true`**. @@ -163,17 +163,8 @@ queue. On ambiguous dispatch, timeout, or agent/write outcome, inspect coordinat state before manually reconciling through this queue. It does not serialize chat/direct/external writers or prove a timed-out hosted invocation stopped. -## Offline checks +## Test ownership -With the pinned IssueLens action files available locally, set -`ISSUELENS_ACTIONS_PATH` to their `.github/actions` directory and run: - -```text -python -m unittest discover -s .github/actions/queue-team-memory/tests -``` - -The focused CI workflow fetches the same immutable shared actions for tests only. -Checks use actual action schemas and generic dispatch payloads, source-ID/read -authentication, `main`/`develop` handling, complete range/PR discovery, and final -receipts with fake API/stream responses. They make no live dispatch, Azure, or -wiki calls. +Team-memory tests are owned and run in [microsoft/IssueLens](https://github.com/microsoft/IssueLens). +Java Pack consumes the pinned shared actions without duplicating their test +suite or adding a local team-memory test workflow. diff --git a/.github/actions/queue-team-memory/tests/test_team_memory.py b/.github/actions/queue-team-memory/tests/test_team_memory.py deleted file mode 100644 index 8a74945f..00000000 --- a/.github/actions/queue-team-memory/tests/test_team_memory.py +++ /dev/null @@ -1,548 +0,0 @@ -import copy -import importlib.util -import io -import json -import os -import subprocess -import tempfile -import unittest -from email.message import Message -from pathlib import Path -from unittest.mock import Mock, patch - -import yaml - -ROOT = Path(__file__).resolve().parents[4] -SHARED = Path(os.environ["ISSUELENS_ACTIONS_PATH"]).resolve() -PIN = "a81d2d96167fc0e69ac631c2edc85f858e693289" -COORDINATOR = "microsoft/vscode-java-pack" - - -def load(path, name): - spec = importlib.util.spec_from_file_location(name, path) - module = importlib.util.module_from_spec(spec) - spec.loader.exec_module(module) - return module - - -def read_yaml(path): - return yaml.load(path.read_text(encoding="utf-8"), Loader=yaml.BaseLoader) - - -invoker = load(SHARED / "issuelens" / "issuelens_action.py", "shared_invoker") -dispatcher = load(SHARED / "queue-team-memory" / "dispatch.py", "shared_dispatcher") -source_workflow = read_yaml(ROOT / ".github" / "workflows" / "team-memory-post-merge.yml") -central_workflow = read_yaml(ROOT / ".github" / "workflows" / "team-memory-coordinator.yml") -source_job = source_workflow["jobs"]["dispatch"] -central_job = central_workflow["jobs"]["reconcile"] -ALLOWED = json.loads(central_job["env"]["ISSUELENS_SOURCE_REPOSITORIES"]) - - -class Response: - def __init__(self, content=b"", status=200): - self.content = io.BytesIO(content) - self.status = status - self.headers = Message() - self.headers["Content-Type"] = "text/event-stream" - - def __enter__(self): - return self - - def __exit__(self, *args): - return False - - def read(self, limit): - return self.content.read(limit) - - def readline(self, limit): - return self.content.readline(limit) - - -class ContractTests(unittest.TestCase): - def setUp(self): - self.temp = tempfile.TemporaryDirectory() - self.addCleanup(self.temp.cleanup) - self.directory = Path(self.temp.name) - self.before, self.merge, self.after, self.tip = [letter * 40 for letter in "daef"] - self.source = "microsoft/vscode-gradle" - self.central = self.project(COORDINATOR) - self.source_project = self.project(self.source) - self.environment = { - "GITHUB_REPOSITORY": COORDINATOR, "GITHUB_EVENT_NAME": "workflow_dispatch", - "GITHUB_REF": "refs/heads/main", "GITHUB_SHA": self.tip, "GITHUB_WORKFLOW_SHA": self.tip, - "GITHUB_WORKFLOW_REF": COORDINATOR + "/.github/workflows/team-memory-coordinator.yml@refs/heads/main", - "GITHUB_ACTOR": "maintainer", "GITHUB_TRIGGERING_ACTOR": "rerunner", - "GITHUB_RUN_ID": "999", "GITHUB_RUN_ATTEMPT": "1", - "GITHUB_EVENT_PATH": str(self.directory / "event.json"), "RUNNER_TEMP": str(self.directory), - "GITHUB_OUTPUT": str(self.directory / "output.txt"), - "GH_TOKEN": "test-coordinator-token", "SOURCE_GH_TOKEN": "test-source-token", - "SOURCE_REPOSITORY": self.source, "SOURCE_REPOSITORIES": json.dumps(ALLOWED), - "SOURCE_RUN_ID": "123456", "SOURCE_RUN_ATTEMPT": "2", - "PUSH_BEFORE": self.before, "PUSH_AFTER": self.after, "DISPATCH_PR": "", - "REQUEST_TYPE": "team-memory", "TASK_INPUT": "", "OUTPUT_MODE": "quiet", "SUMMARY_MODE": "none", - "AGENT_URL": "https://test.services.ai.azure.com/agents/test/protocols/invocations", - "AGENT_SCOPE": "api://test/.default", "DISPATCH_TOKEN": "test-dispatch-token", - "COORDINATOR_REPOSITORY": COORDINATOR, "COORDINATOR_WORKFLOW": "team-memory-coordinator.yml", - "COORDINATOR_REF": "main", - } - self.env_patch = patch.dict(os.environ, self.environment, clear=True) - self.env_patch.start() - self.addCleanup(self.env_patch.stop) - Path(os.environ["GITHUB_EVENT_PATH"]).write_text(json.dumps({"repository": self.central})) - self.source_run = { - "id": 123456, "run_attempt": 2, "event": "push", "path": invoker.DISPATCH_WORKFLOW, - "head_branch": "develop", "head_sha": self.after, "repository": self.source_project, - "head_repository": self.source_project, "actor": {"login": "maintainer"}, - "triggering_actor": {"login": "rerunner"}, - } - self.branch = {"name": "develop", "commit": {"sha": self.tip}} - self.ancestry = self.comparison(self.after, 1) - self.inventory = self.comparison(self.before, 2, [self.merge, self.after]) - self.pull = { - "number": 27, "merged": True, "state": "closed", "merge_commit_sha": self.merge, - "merged_at": "2026-10-09T00:00:00Z", "base": {"ref": "develop", "repo": self.source_project}, - } - self.graphql = self.associations([self.merge, self.after]) - self.target_workflow = {"id": 789, "path": ".github/workflows/team-memory-coordinator.yml", "state": "active"} - self.target_branch = {"name": "main", "commit": {"sha": self.tip}} - self.submit_response = Response() - self.requests = [] - self.opener = Mock() - self.opener.open.side_effect = self.respond - self.patches = [ - patch("urllib.request.build_opener", return_value=self.opener), - patch.object(invoker.subprocess, "check_output", return_value="test-azure-token"), - patch("sys.stdout", new_callable=io.StringIO), - ] - _, self.azure, self.log = [item.start() for item in self.patches] - for item in self.patches: - self.addCleanup(item.stop) - - def project(self, name): - branch = "develop" if name in {"microsoft/vscode-gradle", "microsoft/build-server-for-gradle"} else "main" - return {"id": ALLOWED[name], "full_name": name, "default_branch": branch, "visibility": "public"} - - def comparison(self, before, count, shas=()): - return {"base_commit": {"sha": before}, "merge_base_commit": {"sha": before}, - "status": "ahead", "ahead_by": count, "behind_by": 0, "total_commits": count, - "commits": [{"sha": sha} for sha in shas]} - - def associations(self, shas): - result = {"databaseId": self.source_project["id"], "nameWithOwner": self.source, - "defaultBranchRef": {"name": self.source_project["default_branch"], "target": {"oid": self.tip}}} - for index, sha in enumerate(shas): - node = {"number": 27, "state": "MERGED", "merged": True, "mergedAt": self.pull["merged_at"], - "baseRefName": self.source_project["default_branch"], - "baseRepository": {"databaseId": self.source_project["id"], "nameWithOwner": self.source}, - "mergeCommit": {"oid": self.merge}} - result[f"c{index}"] = {"oid": sha, "associatedPullRequests": { - "totalCount": 1, "pageInfo": {"hasNextPage": False}, "nodes": [node]}} - return {"data": {"repository": result}} - - def select_source(self, name): - self.source = name - self.source_project = self.project(name) - os.environ["SOURCE_REPOSITORY"] = name - self.source_run.update(repository=self.source_project, head_repository=self.source_project, - head_branch=self.source_project["default_branch"]) - self.branch["name"] = self.source_project["default_branch"] - self.pull["base"] = {"ref": self.source_project["default_branch"], "repo": self.source_project} - self.graphql = self.associations([self.merge, self.after]) - - def respond(self, request, timeout): - self.requests.append(request) - path = request.full_url.removeprefix("https://api.github.com") - if request.full_url == os.environ["AGENT_URL"]: - return self.submit_response - if path.endswith("/dispatches"): - return Response(status=204) - responses = { - f"/repos/{COORDINATOR}": self.central, f"/repos/{self.source}": self.source_project, - f"/repos/{COORDINATOR}/actions/workflows/team-memory-coordinator.yml": self.target_workflow, - f"/repos/{COORDINATOR}/branches/main": self.target_branch, - f"/repos/{self.source}/actions/runs/123456/attempts/2": self.source_run, - f"/repos/{self.source}/branches/{self.source_project['default_branch']}": self.branch, - f"/repos/{self.source}/compare/{self.after}...{self.tip}?per_page=1&page=2": self.ancestry, - f"/repos/{self.source}/compare/{self.before}...{self.after}?per_page=100&page=1": self.inventory, - f"/repos/{self.source}/pulls/27": self.pull, "/graphql": self.graphql, - } - self.assertIn(path, responses, "Unexpected network path") - return Response(json.dumps(responses[path]).encode()) - - def outputs(self): - path = Path(os.environ["GITHUB_OUTPUT"]) - return dict(line.split("=", 1) for line in path.read_text().splitlines()) if path.exists() else {} - - def preflight(self): - invoker.run("preflight") - self.azure.assert_not_called() - self.assertFalse(any(request.get_method() == "POST" and not request.full_url.endswith("/graphql") - for request in self.requests)) - return json.loads(Path(self.outputs()["request-path"]).read_text()) if self.outputs().get("eligible") == "true" else None - - def dispatch_inputs(self): - content = source_job["steps"][0]["with"]["workflow-inputs"] - replacements = {"github.repository": self.source, "github.run_id": "123456", "github.run_attempt": "2", - "github.event.before": self.before, "github.sha": self.after} - for name, value in replacements.items(): - content = content.replace("${{ " + name + " }}", value) - return content - - def test_source_main_and_develop_dispatch_scalar_inputs_to_central_main(self): - for name in (COORDINATOR, "microsoft/vscode-gradle"): - with self.subTest(source=name): - self.select_source(name) - self.requests.clear() - os.environ["WORKFLOW_INPUTS"] = self.dispatch_inputs() - dispatcher.run() - posted, = [request for request in self.requests if request.get_method() == "POST"] - self.assertEqual(json.loads(posted.data), {"ref": "main", "inputs": { - "source_repository": name, "source_run_id": "123456", "source_run_attempt": "2", - "push_before": self.before, "push_after": self.after, - }}) - self.assertEqual(len(self.requests), 4) - self.assertTrue(all(request.get_header("Authorization") == "Bearer test-dispatch-token" - for request in self.requests)) - self.azure.assert_not_called() - - def test_generic_dispatch_payload_drives_shared_invocation_without_artifacts(self): - os.environ["WORKFLOW_INPUTS"] = self.dispatch_inputs() - dispatcher.run() - inputs = json.loads(self.requests[-1].data)["inputs"] - os.environ.update({name.upper(): value for name, value in inputs.items()}) - self.requests.clear() - metadata = self.preflight()["metadata"] - self.assertEqual(metadata["repository"], self.source) - self.assertEqual(metadata["required_wiki_repository"], COORDINATOR) - self.assertEqual(metadata["range_origin"], "authorized-reconciliation") - self.assertFalse(any("artifact" in request.full_url for request in self.requests)) - - def test_dispatch_ambiguous_post_has_no_retry_or_completion_claim(self): - os.environ["WORKFLOW_INPUTS"] = self.dispatch_inputs() - original = self.respond - - def fail_post(request, timeout): - if request.get_method() == "POST": - self.requests.append(request) - raise OSError("PRIVATE_TRANSPORT_DETAIL") - return original(request, timeout) - - self.opener.open.side_effect = fail_post - with self.assertRaisesRegex(SystemExit, "outcome is unknown"): - dispatcher.run() - self.assertEqual(sum(request.get_method() == "POST" for request in self.requests), 1) - self.assertNotIn("PRIVATE_TRANSPORT_DETAIL", self.log.getvalue()) - self.assertEqual(self.outputs(), {}) - - def test_dispatch_empty_token_and_invalid_payloads_fail_before_network(self): - for changes in ({"DISPATCH_TOKEN": ""}, {"WORKFLOW_INPUTS": '{"run":1}'}, - {"WORKFLOW_INPUTS": '{"run":"1","run":"2"}'}, - {"WORKFLOW_INPUTS": "\u20ac" * 22000}, {"COORDINATOR_WORKFLOW": "../bad.yml"}): - with self.subTest(changes=changes), patch.dict(os.environ, changes), self.assertRaises(SystemExit): - dispatcher.run() - self.opener.open.assert_not_called() - - def test_inactive_target_workflow_or_wrong_branch_cannot_dispatch(self): - os.environ["WORKFLOW_INPUTS"] = self.dispatch_inputs() - self.target_workflow["state"] = "disabled_manually" - with self.assertRaisesRegex(SystemExit, "workflow is inactive"): - dispatcher.run() - self.target_workflow["state"] = "active" - self.target_branch["name"] = "develop" - with self.assertRaisesRegex(SystemExit, "branch identity"): - dispatcher.run() - self.assertFalse(any(request.get_method() == "POST" for request in self.requests)) - - def test_all_eight_trusted_ids_use_current_source_default_branch(self): - for name, identity in ALLOWED.items(): - with self.subTest(source=name): - self.select_source(name) - metadata = self.preflight()["metadata"] - self.assertEqual(metadata["repository_id"], identity) - self.assertEqual(metadata["base_ref"], self.source_project["default_branch"]) - self.assertEqual(metadata["coordinator_repository"], COORDINATOR) - self.assertEqual(metadata["required_wiki_repository"], COORDINATOR) - - def test_source_and_coordinator_identity_and_read_credentials_stay_separate(self): - envelope = self.preflight() - metadata = envelope["metadata"] - self.assertEqual(metadata["workflow_sha"], self.after) - self.assertEqual(metadata["coordinator_workflow_sha"], self.tip) - self.assertEqual(metadata["run_id"], 123456) - self.assertEqual(metadata["coordinator_run_id"], 999) - self.assertEqual(self.requests[0].get_header("Authorization"), "Bearer test-coordinator-token") - self.assertTrue(all(request.get_header("Authorization") == "Bearer test-source-token" - for request in self.requests[1:])) - self.assertNotIn("test-source-token", json.dumps(envelope)) - self.assertIn("This never overrides repository policy", envelope["request"]["input"]) - self.assertIn("Do not modify source or issues", envelope["request"]["input"]) - - def test_requested_before_is_an_ancestor_selection_not_original_push_proof(self): - requested = "c" * 40 - os.environ["PUSH_BEFORE"] = requested - self.before = requested - self.inventory = self.comparison(requested, 2, [self.merge, self.after]) - envelope = self.preflight() - self.assertEqual(envelope["metadata"]["push_before"], requested) - self.assertIn("not an attestation of the original push boundary", envelope["request"]["input"]) - self.assertEqual(set(invoker.SOURCE_INPUTS), {"SOURCE_RUN_ID", "SOURCE_RUN_ATTEMPT", "PUSH_BEFORE", "PUSH_AFTER"}) - - def test_unknown_source_mixed_inputs_and_empty_source_token_fail_before_network(self): - for changes in ({"SOURCE_REPOSITORY": "microsoft/IssueLens"}, {"SOURCE_RUN_ID": ""}, - {"SOURCE_RUN_ATTEMPT": "0"}, {"PUSH_BEFORE": ""}, {"PUSH_AFTER": "short"}, - {"PUSH_BEFORE": self.after}, {"DISPATCH_PR": "27"}, - {"SOURCE_GH_TOKEN": ""}, {"SOURCE_GH_TOKEN": "bad\ncredential"}): - with self.subTest(changes=changes), patch.dict(os.environ, changes), self.assertRaises(SystemExit): - self.preflight() - self.opener.open.assert_not_called() - - def test_changed_source_id_canonical_name_or_private_visibility_is_rejected(self): - original = copy.deepcopy(self.source_project) - for changes in ({"id": original["id"] + 1}, {"full_name": "microsoft/renamed"}, - {"visibility": "private"}, {"visibility": "internal"}): - with self.subTest(changes=changes): - self.source_project = {**original, **changes} - with self.assertRaises(SystemExit): - self.preflight() - self.assertFalse(any("/actions/runs/" in request.full_url for request in self.requests)) - - def test_forged_source_run_attempt_workflow_head_or_branch_cannot_invoke(self): - original = copy.deepcopy(self.source_run) - for changes in ({"id": 123457}, {"run_attempt": 1}, {"event": "pull_request"}, - {"path": ".github/workflows/untrusted.yml"}, {"head_sha": self.tip}, - {"head_branch": "main"}, {"head_repository": self.central}): - with self.subTest(changes=changes): - self.source_run = {**original, **changes} - with self.assertRaisesRegex(SystemExit, "Source run does not match"): - self.preflight() - self.assertEqual(self.outputs(), {}) - - def test_wrong_coordinator_workflow_or_nondefault_ref_is_rejected(self): - for changes in ({"GITHUB_REF": "refs/heads/feature"}, { - "GITHUB_WORKFLOW_REF": COORDINATOR + "/.github/workflows/untrusted.yml@refs/heads/main"}): - with self.subTest(changes=changes), patch.dict(os.environ, changes), self.assertRaises(SystemExit): - self.preflight() - self.assertEqual(self.outputs(), {}) - - def test_divergent_head_or_current_source_tip_race_is_rejected(self): - self.ancestry["behind_by"] = 1 - with self.assertRaisesRegex(SystemExit, "complete fast-forward"): - self.preflight() - self.ancestry["behind_by"] = 0 - self.graphql["data"]["repository"]["defaultBranchRef"]["target"]["oid"] = self.merge - with self.assertRaisesRegex(SystemExit, "changed during reconciliation"): - self.preflight() - - def test_truncated_duplicate_divergent_or_missing_head_range_fails(self): - original = copy.deepcopy(self.inventory) - for changes in ({"commits": [{"sha": self.after}]}, - {"commits": [{"sha": self.after}, {"sha": self.after}]}, - {"commits": [{"sha": self.merge}, {"sha": self.tip}]}, - {"status": "diverged"}, {"total_commits": 1001}, {"total_commits": 0}): - with self.subTest(changes=changes): - self.inventory = {**original, **changes} - with self.assertRaises(SystemExit): - self.preflight() - self.assertEqual(self.outputs(), {}) - - def test_complete_1000_commit_pagination_and_partial_final_page(self): - for count in (102, 1000): - with self.subTest(count=count): - shas = [f"{number:040x}" for number in range(1, count)] + [self.after] - pages = [self.comparison(self.before, count, shas[start:start + 100]) for start in range(0, count, 100)] - with patch.object(invoker, "github_read", side_effect=pages) as read: - actual = invoker.read_reconciliation_inventory(self.source, self.before, self.after, float("inf")) - self.assertEqual(actual, shas) - self.assertEqual(read.call_count, len(pages)) - self.assertTrue(all(call.kwargs["token"] == "test-source-token" for call in read.call_args_list)) - self.assertTrue(read.call_args.args[0].endswith(f"page={len(pages)}")) - - def test_missing_second_range_page_and_incomplete_pr_associations_fail(self): - shas = [f"{number:040x}" for number in range(1, 102)] + [self.after] - pages = [self.comparison(self.before, len(shas), shas[:100]), - self.comparison(self.before, len(shas), shas[100:-1])] - with patch.object(invoker, "github_read", side_effect=pages), self.assertRaisesRegex(ValueError, "pagination"): - invoker.read_reconciliation_inventory(self.source, self.before, self.after, float("inf")) - self.graphql["data"]["repository"]["c0"]["associatedPullRequests"]["pageInfo"]["hasNextPage"] = True - with self.assertRaisesRegex(SystemExit, "associations are incomplete"): - self.preflight() - - def test_rebase_merge_rest_validation_and_no_merged_pr_skip(self): - for name in ("c0", "c1"): - self.graphql["data"]["repository"][name]["associatedPullRequests"]["nodes"][0]["mergeCommit"] = None - self.assertEqual(self.preflight()["metadata"]["pull_requests"][0]["merge_commit_sha"], self.merge) - self.assertEqual(sum(request.full_url.endswith("/pulls/27") for request in self.requests), 1) - for name in ("c0", "c1"): - self.graphql["data"]["repository"][name]["associatedPullRequests"] = { - "totalCount": 0, "pageInfo": {"hasNextPage": False}, "nodes": []} - Path(os.environ["GITHUB_OUTPUT"]).unlink() - self.assertIsNone(self.preflight()) - self.assertEqual(self.outputs(), {"eligible": "false", "status": "skipped", "skip-reason": "no_merged_pull_requests"}) - - def test_manual_pr_uses_same_shared_invoker_and_source_default_branch(self): - os.environ.update({**{name: "" for name in invoker.SOURCE_INPUTS}, "DISPATCH_PR": "27"}) - metadata = self.preflight()["metadata"] - self.assertEqual(metadata["base_ref"], "develop") - self.assertEqual(metadata["pull_number"], 27) - self.assertEqual(metadata["required_wiki_repository"], COORDINATOR) - self.assertFalse(any("/actions/runs/" in request.full_url for request in self.requests)) - self.pull["merged"] = False - with self.assertRaisesRegex(SystemExit, "not merged"): - self.preflight() - - def result(self): - return {"source_repository": self.source, "push_before": self.before, "push_after": self.after, - "status": "no-change", "wiki_repository": COORDINATOR, "wiki_sha": "b" * 40, - "reason": "Verified source and wiki", "results": [ - {"pull_number": 27, "merge_commit_sha": self.merge, "status": "no-change", "reason": "Verified"}]} - - def submit(self, result, done=True): - os.environ["REQUEST_PATH"] = self.outputs()["request-path"] - stream = "data: " + json.dumps({"type": "assistant.message", "data": {"content": json.dumps(result)}}) + "\n\n" - if done: - stream += 'event: done\ndata: {"invocation_id":"test","session_id":"test"}\n\n' - self.submit_response = Response(stream.encode()) - invoker.run("submit") - - def test_completed_stream_requires_valid_receipt_with_shared_wiki_and_sha(self): - self.preflight() - self.submit(self.result()) - self.assertEqual(self.outputs()["status"], "no-change") - self.assertEqual(self.outputs()["wiki-repository"], COORDINATOR) - self.assertEqual(sum(request.full_url == os.environ["AGENT_URL"] for request in self.requests), 1) - self.assertEqual(self.opener.open.call_args.kwargs["timeout"], 300) - - def test_wrong_wiki_source_missing_pr_or_success_shaped_status_cannot_succeed(self): - self.preflight() - original = self.result() - for changes in ({"wiki_repository": "microsoft/IssueLens"}, {"wiki_sha": None}, - {"source_repository": COORDINATOR}, {"results": []}, {"status": "completed"}): - with self.subTest(changes=changes), self.assertRaises(SystemExit): - self.submit({**original, **changes}) - self.assertNotIn("status", self.outputs()) - - def test_incomplete_stream_is_unknown_and_never_retried(self): - self.preflight() - with self.assertRaisesRegex(SystemExit, "completion event; outcome unknown"): - self.submit(self.result(), done=False) - self.assertEqual(sum(request.full_url == os.environ["AGENT_URL"] for request in self.requests), 1) - self.assertNotIn("status", self.outputs()) - - def test_partial_publication_keeps_receipt_but_fails_batch(self): - connection = self.graphql["data"]["repository"]["c1"]["associatedPullRequests"] - connection["nodes"].append({**copy.deepcopy(connection["nodes"][0]), "number": 28, - "mergeCommit": {"oid": self.after}}) - connection["totalCount"] = 2 - self.preflight() - result = self.result() - result["status"] = "partial" - result["results"].append({"pull_number": 28, "merge_commit_sha": self.after, - "status": "needs-review", "reason": "Incomplete evidence"}) - with self.assertRaisesRegex(SystemExit, "batch incomplete"): - self.submit(result) - self.assertEqual(self.outputs()["status"], "partial") - self.assertEqual(self.outputs()["wiki-repository"], COORDINATOR) - - -class WiringTests(unittest.TestCase): - def test_workflows_use_latest_real_input_schemas_and_only_shared_invocation(self): - queue = read_yaml(SHARED / "queue-team-memory" / "action.yml") - invoke = read_yaml(SHARED / "issuelens" / "action.yml") - dispatch, = source_job["steps"] - self.assertEqual(dispatch["uses"], "microsoft/IssueLens/.github/actions/queue-team-memory@" + PIN) - self.assertTrue(set(dispatch["with"]).issubset(queue["inputs"])) - selection, auth, invocation = central_job["steps"] - self.assertEqual(invocation["uses"], "microsoft/IssueLens/.github/actions/issuelens@" + PIN) - self.assertTrue(set(invocation["with"]).issubset(invoke["inputs"])) - self.assertEqual(invocation["with"]["source-repositories"], "${{ env.ISSUELENS_SOURCE_REPOSITORIES }}") - for name in ("source_repository", "source_run_id", "source_run_attempt", "push_before", "push_after"): - self.assertEqual(invocation["with"][name.replace("_", "-")], "${{ inputs." + name + " }}") - self.assertNotIn("continue-on-error", invocation) - self.assertNotIn("run", invocation) - preflight, login, submit = invoke["runs"]["steps"] - self.assertEqual(preflight["env"]["SOURCE_GH_TOKEN"], "${{ inputs.source-github-token }}") - self.assertTrue(all(step["if"] == "steps.preflight.outputs.eligible == 'true'" for step in (login, submit))) - self.assertEqual(len(queue["runs"]["steps"]), 1) - self.assertEqual(queue["runs"]["steps"][0]["run"], 'python3 -I "$GITHUB_ACTION_PATH/dispatch.py"') - - def test_source_gates_and_one_central_queue_cover_manual_and_automatic_requests(self): - self.assertEqual(source_workflow["on"], {"push": {"branches": ["main"]}}) - self.assertEqual(set(central_workflow["on"]), {"workflow_dispatch"}) - self.assertEqual(central_workflow["concurrency"], { - "group": "issuelens-team-memory-wiki-microsoft-vscode-java-pack", "queue": "max", "cancel-in-progress": "false"}) - self.assertNotIn("concurrency", source_workflow) - for workflow, job in ((source_workflow, source_job), (central_workflow, central_job)): - self.assertEqual(workflow["permissions"], {}) - self.assertIn("ISSUELENS_TEAM_MEMORY_COORDINATOR_ENABLED == 'true'", job["if"]) - self.assertIn("github.event.repository.default_branch", job["if"]) - for flag in ("created", "deleted", "forced"): - self.assertIn(f"github.event.{flag} == false", source_job["if"]) - self.assertIn("github.workflow_sha == github.sha", source_job["if"]) - self.assertEqual(source_job["permissions"], {"contents": "read", "actions": "write"}) - self.assertEqual(central_job["permissions"], { - "contents": "read", "actions": "read", "pull-requests": "read", "id-token": "write"}) - self.assertEqual(central_job["timeout-minutes"], "30") - inputs = central_workflow["on"]["workflow_dispatch"]["inputs"] - self.assertEqual(set(inputs), {"source_repository", "source_run_id", "source_run_attempt", - "push_before", "push_after", "pull_request_number"}) - self.assertIn("not an attested original push boundary", inputs["push_before"]["description"]) - - def test_trusted_allowlist_selection_precedes_single_source_read_authentication(self): - self.assertEqual(ALLOWED, { - COORDINATOR: 104967329, "microsoft/vscode-java-debug": 102584737, - "microsoft/vscode-java-test": 110522074, "microsoft/vscode-java-dependency": 129053101, - "microsoft/vscode-maven": 116921700, "microsoft/vscode-gradle": 216314492, - "microsoft/java-debug": 102583752, "microsoft/build-server-for-gradle": 626812412}) - selection, auth, invocation = central_job["steps"] - self.assertIn("'has($repository)'", selection["run"]) - self.assertIn('exit 1', selection["run"]) - self.assertNotIn("${{", selection["run"]) - self.assertEqual(auth["if"], "steps.selection.outputs.external == 'true'") - self.assertEqual(auth["with"]["repositories"], "${{ steps.selection.outputs.source-name }}") - self.assertEqual(auth["with"]["owner"], "microsoft") - self.assertEqual({key: value for key, value in auth["with"].items() if key.startswith("permission-")}, - {"permission-actions": "read", "permission-contents": "read", "permission-pull-requests": "read"}) - self.assertEqual(invocation["with"]["source-github-token"], - "${{ steps.selection.outputs.external == 'false' && github.token || steps.source-token.outputs.token }}") - self.assertEqual(invocation["with"]["output-mode"], "activity") - self.assertEqual(invocation["with"]["summary-mode"], "status") - - @unittest.skipIf(os.name == "nt", "Receiver selection runs on Ubuntu") - def test_actual_source_selection_script_restricts_scoped_app_authentication(self): - script = central_job["steps"][0]["run"] - with tempfile.TemporaryDirectory() as directory: - output = Path(directory) / "output.txt" - for repository in (*ALLOWED, "", "microsoft/IssueLens", "microsoft/vscode-spring-initializr", - "fork/vscode-java-pack", "microsoft/vscode-java-pack\ninjected=true"): - with self.subTest(source=repository): - output.unlink(missing_ok=True) - result = subprocess.run( - ["bash", "-e", "-c", script], capture_output=True, text=True, check=False, - env={**os.environ, "ISSUELENS_SOURCE_REPOSITORIES": json.dumps(ALLOWED), - "SOURCE_REPOSITORY": repository, "GITHUB_OUTPUT": str(output)}, - ) - if repository in ALLOWED: - self.assertEqual(result.returncode, 0, result.stderr) - external = "false" if repository == COORDINATOR else "true" - self.assertEqual(output.read_text().splitlines(), [ - f"source-name={repository.removeprefix('microsoft/')}", f"external={external}"]) - else: - self.assertEqual(result.returncode, 1) - self.assertIn("::error::Source repository is not allowlisted", result.stdout) - self.assertFalse(output.exists()) - - def test_no_runtime_artifacts_local_invoker_or_client_checkout(self): - for name in ("team-memory-post-merge.yml", "team-memory-coordinator.yml"): - content = (ROOT / ".github" / "workflows" / name).read_text() - for forbidden in ("artifact", "actions/checkout", "issuelens_action.py", "team_memory.py", - "ISSUELENS_CLIENT_PATH", "azure/login@"): - self.assertNotIn(forbidden, content) - self.assertFalse((Path(__file__).parents[1] / "team_memory.py").exists()) - self.assertFalse((Path(__file__).parents[1] / "action.yml").exists()) - tests = read_yaml(ROOT / ".github" / "workflows" / "team-memory-tests.yml") - self.assertEqual(tests["jobs"]["test"]["steps"][1]["with"]["ref"], PIN) - self.assertIn(PIN, (Path(__file__).parents[1] / "README.md").read_text()) - - -if __name__ == "__main__": - unittest.main() diff --git a/.github/workflows/team-memory-tests.yml b/.github/workflows/team-memory-tests.yml deleted file mode 100644 index 9e486301..00000000 --- a/.github/workflows/team-memory-tests.yml +++ /dev/null @@ -1,41 +0,0 @@ -name: Java team-memory contract checks - -on: - pull_request: - paths: - - .github/actions/queue-team-memory/** - - .github/workflows/team-memory*.yml - push: - branches: [main] - paths: - - .github/actions/queue-team-memory/** - - .github/workflows/team-memory*.yml - -permissions: {} - -jobs: - test: - runs-on: ubuntu-latest - permissions: - contents: read - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - persist-credentials: false - - - name: Load pinned shared actions for offline contract tests only - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - repository: microsoft/IssueLens - ref: a81d2d96167fc0e69ac631c2edc85f858e693289 - path: .issuelens-client - persist-credentials: false - sparse-checkout: | - /.github/actions/issuelens/ - /.github/actions/queue-team-memory/ - sparse-checkout-cone-mode: false - - - name: Check dispatch, source authorization, discovery, and receipts offline - env: - ISSUELENS_ACTIONS_PATH: .issuelens-client/.github/actions - run: python3 -m unittest discover -s .github/actions/queue-team-memory/tests From 4312340df120299c69ddbb7365158e1881fe26dc Mon Sep 17 00:00:00 2001 From: Changyong Gong Date: Sat, 10 Oct 2026 13:27:28 +0800 Subject: [PATCH 5/5] Remove local queue-team-memory assets Keep only the remote IssueLens workflow consumers. Remove the remaining local setup README; prerequisites remain in the PR description. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- .github/actions/queue-team-memory/README.md | 170 -------------------- 1 file changed, 170 deletions(-) delete mode 100644 .github/actions/queue-team-memory/README.md diff --git a/.github/actions/queue-team-memory/README.md b/.github/actions/queue-team-memory/README.md deleted file mode 100644 index 11980991..00000000 --- a/.github/actions/queue-team-memory/README.md +++ /dev/null @@ -1,170 +0,0 @@ -# Java team-memory coordinator - -Java Pack hosts the single queue for the `microsoft/vscode-java-pack` wiki. -The [push caller](../../workflows/team-memory-post-merge.yml) invokes only the -standalone IssueLens dispatch action with five scalar reconciliation inputs. -The [coordinator](../../workflows/team-memory-coordinator.yml) invokes the shared -IssueLens invocation action directly. Both actions are pinned to current upstream -`a81d2d96167fc0e69ac631c2edc85f858e693289`, including the merged shared-action work -in [microsoft/IssueLens#51](https://github.com/microsoft/IssueLens/pull/51). -There is no unmerged prerequisite or hosted runtime deployment for action reuse. - -Manual PR reconciliation also runs through this central queue. There is no -source preparation script, checkout, artifact upload/download, or local -invoker/provenance adapter. This directory retains setup guidance only. -Shared actions own transport, source validation, discovery, -Azure OIDC login, invocation, and final receipt checks. - -Both workflows require **`ISSUELENS_TEAM_MEMORY_COORDINATOR_ENABLED=true`**. -This is a new, disabled-by-default repository variable: the existing -`ISSUELENS_TEAM_MEMORY_ENABLED` does not enable this rollout. This PR does not -change variables, credentials, App installations, Azure access, or agent -deployments. Verify the prerequisites below before a separately authorized -enablement. Issue-loop and direct-task consumers are unchanged. - -## Source trust and scope - -The trusted `ISSUELENS_SOURCE_REPOSITORIES` JSON in the coordinator workflow maps -canonical source names to immutable numeric repository IDs. It is never supplied -by a dispatch input. The shared invocation action validates canonical identity, -public-to-public cross-repository access, and each source's current default -branch against its source run/PR. These sources currently configure -`instructions.team_memory.wiki_repository: microsoft/vscode-java-pack`: - -| Source | Default branch | Reviewed policy snapshot | -| --- | --- | --- | -| `microsoft/vscode-java-pack` | `main` | [0823dbde](https://github.com/microsoft/vscode-java-pack/blob/0823dbde1947de111c39890fc095f55c0ff0d258/.github/issuelens.yml) | -| `microsoft/vscode-java-debug` | `main` | [3705df0d](https://github.com/microsoft/vscode-java-debug/blob/3705df0d0b1215b044e759647e50e4caa231e5b6/.github/issuelens.yml) | -| `microsoft/vscode-java-test` | `main` | [cff6fdb4](https://github.com/microsoft/vscode-java-test/blob/cff6fdb4b5e3221a4e88473d2f90e1fd58f1061c/.github/issuelens.yml) | -| `microsoft/vscode-java-dependency` | `main` | [eb3fba04](https://github.com/microsoft/vscode-java-dependency/blob/eb3fba04f2763416b25833d92df4fb0210a9d668/.github/issuelens.yml) | -| `microsoft/vscode-maven` | `main` | [eee9eb0b](https://github.com/microsoft/vscode-maven/blob/eee9eb0b49b8b5bdd4a479646d700340a2b37cbd/.github/issuelens.yml) | -| `microsoft/vscode-gradle` | `develop` | [1a544c91](https://github.com/microsoft/vscode-gradle/blob/1a544c917399ead6c23251301a041e505c0a6926/.github/issuelens.yml) | -| `microsoft/java-debug` | `main` | [bcae605a](https://github.com/microsoft/java-debug/blob/bcae605a6240f11b447163a614e215721c11b107/.github/issuelens.yml) | -| `microsoft/build-server-for-gradle` | `develop` | [ab37cca9](https://github.com/microsoft/build-server-for-gradle/blob/ab37cca97693c60e90d257ce9eb6e9575da7e6f1/.github/issuelens.yml) | - -The two Spring candidates remain excluded: their policies were absent/unverified. -An allowlist entry is not a source migration or -authorization to ignore policy. Only Java Pack's caller changes in this PR. -Other repositories must adopt the shared IssueLens action in separate reviewed PRs. -Renames, transfers, branch changes, or private sources require another review. - -The hosted agent must resolve each source's policy and privacy constraints, -verify that its validated destination is this wiki, and stop rather than -override a conflicting policy. Requests explicitly require this destination; -the pinned client's final receipt validator rejects any other wiki. Neither -the standalone dispatch action nor the coordinator's allowlist grants wiki write access. - -## Future external callers - -Use `.github/workflows/team-memory-post-merge.yml` on the source's **actual -default branch**, gated by its own opt-in variable, with job permissions -`contents: read` and `actions: write`. No source checkout or copied Python is -needed for a remote reference: - -```yaml -- name: Queue Java team-memory update - uses: microsoft/IssueLens/.github/actions/queue-team-memory@a81d2d96167fc0e69ac631c2edc85f858e693289 - with: - coordinator-repository: microsoft/vscode-java-pack - coordinator-workflow: team-memory-coordinator.yml - coordinator-ref: main - dispatch-token: ${{ steps.dispatch-token.outputs.token }} - workflow-inputs: >- - {"source_repository":"${{ github.repository }}", - "source_run_id":"${{ github.run_id }}", - "source_run_attempt":"${{ github.run_attempt }}", - "push_before":"${{ github.event.before }}", - "push_after":"${{ github.sha }}"} -``` - -Do not use a moving action branch. `coordinator-ref: main` is the destination -branch, independently of a source default branch such as `develop`. Source -workflows must gate on the actual default branch, `github.workflow_sha == -github.sha`, and non-created/non-deleted/non-forced pushes, as Java Pack's caller -does. The generic action does not infer or attest source fields; the caller owns -the JSON input mapping. - -The caller must obtain -`steps.dispatch-token.outputs.token` using a **dedicated dispatch GitHub App**, -restricted to Java Pack with **Actions write and Contents read** (the latter -validates the target branch). The source-read token belongs only to the receiving -invocation action, not the generic dispatch action. An explicitly empty -dispatch token fails; it never falls back. -Do not distribute the hosted IssueLens App's private key to source workflows. -A source repository's `GITHUB_TOKEN` remains repository-scoped regardless of -its `actions: write` permission. Java Pack's own caller uses its token for the -target dispatch without new credentials. - -## Coordinator prerequisites - -- Existing central Azure OIDC configuration and secrets: `AZURE_CLIENT_ID`, - `AZURE_TENANT_ID`, `AZURE_SUBSCRIPTION_ID`, `ISSUELENS_AGENT_URL`, and - `ISSUELENS_AGENT_SCOPE`. Their existing names are preserved; secret presence - does not establish that the identity or endpoint is ready. -- For external sources, a separate read-only GitHub App installation covering - the selected allowlisted source, with **Actions read, Contents read, and - Pull requests read**. Configure central secrets - `ISSUELENS_SOURCE_READ_APP_CLIENT_ID` and - `ISSUELENS_SOURCE_READ_APP_PRIVATE_KEY`. The workflow mints an ephemeral, - single-source token only after checking the trusted source-name map and revokes - it at job end. The invocation action independently validates source identity, - run/head, current branch ancestry, and the complete range before Azure login. - Java Pack requests use its own `GITHUB_TOKEN` instead. Source-read - and dispatch installations may differ and must mint separate tokens. -- The existing hosted IssueLens identity must independently have permission - to read the source and publish to the Java Pack wiki, with its normal - policy/privacy and wiki snapshot-precondition enforcement. Do not broaden - its source-code/issue write permissions for this feature. - -The new App secrets are setup prerequisites, not assumed existing credentials. -Verify the central default-branch `workflow_dispatch` Azure OIDC subject and -endpoint access, not just the old source-push identity. No live invocation is -required to validate these repository changes. - -## Verification and outcomes - -Range reconciliation supplies `source_repository`, `source_run_id`, -`source_run_attempt`, `push_before`, and `push_after`. Manual dispatch supplies -`source_repository` and one `pull_request_number`, with all four range fields -empty. Mixed or incomplete requests fail before invocation. -All runs use the same fixed wiki concurrency group, `queue: max`, and -`cancel-in-progress: false`, including validation, discovery, invocation, and -final receipt verification. GitHub supports one active run and at most 100 -pending runs in this queue; it is not an unlimited or guaranteed-delivery queue. -See GitHub's [concurrency documentation](https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/control-workflow-concurrency) -for the `queue: max` contract. - -The standalone dispatcher validates only the target repository, active workflow, -and target branch, then attempts one bounded POST. It transports caller-owned -strings, not commit inventories, bodies, code, or credentials. - -**This authorizes reconciliation of an ancestor range; it does not prove the -original push boundary.** The authenticated source run attests `push_after`/head, -source repository ID, default branch, and exact -`.github/workflows/team-memory-post-merge.yml` path. It cannot establish the -original `push_before` or push flags. The shared invocation action validates -`push_before` as an ancestor, `push_after` as an ancestor of the current source -tip, and retrieves the complete paginated range centrally (up to 1,000 commits, -100 PRs, and a three-minute discovery budget). It rejects truncated/duplicate -inventories, incomplete PR associations, changed identities/tips, and divergent -ranges. Requests explicitly mark `range_origin: authorized-reconciliation` and -require paged PR/source evidence and current-wiki comparison. - -Requests preserve source and coordinator identities separately, require -current-source/wiki comparison, and authorize wiki knowledge writes only. -A successful SSE stream or accepted dispatch is **not successful publication**. -The pinned client validates every PR outcome and the wiki identity/SHA; -partial, failed, needs-review, missing, or wrong-wiki receipts fail the job. -No merged PRs is an explicit preflight skip before Azure login. - -There is one dispatch POST and one agent submission, without automatic retries. -An accepted dispatch does not prove coordinator execution or admission to the -queue. On ambiguous dispatch, timeout, or agent/write outcome, inspect coordinator runs and confirmed wiki -state before manually reconciling through this queue. It does not serialize -chat/direct/external writers or prove a timed-out hosted invocation stopped. - -## Test ownership - -Team-memory tests are owned and run in [microsoft/IssueLens](https://github.com/microsoft/IssueLens). -Java Pack consumes the pinned shared actions without duplicating their test -suite or adding a local team-memory test workflow.