Skip to content

test(integration): make the pinned process-evidence interface executable - #97

Merged
stacknil merged 5 commits into
mainfrom
test/process-evidence-integration-contract
Oct 20, 2026
Merged

stacknil merged 5 commits into
mainfrom
test/process-evidence-integration-contract

Conversation

@stacknil

@stacknil stacknil commented Oct 20, 2026 •

Copy link
Copy Markdown
Owner

Summary

Make the saved process-evidence interface executable: a pinned public fixture, 23 offline consumer contract cases, and a standalone runner that executes the actual producer and consumer.

Why

The October 19 case study describes a successful private execution. A reviewer needs a public regression gate and a fresh reproduction path that requires no private receipts. This suite is later work, based on consumer de542cb3cadba776c8e759f786152f9222fdfdb7 and producer 1d31d4287036cf2bbd3aeb8059d7d332bedef598.

Behavior change

  • Layer A exercises the real loader, normalization, half-open windows, eleven-feature contract, rules, CLI, schemas and fingerprints. Controls cover required fields, malformed JSON/timestamps, equivalent UTC instants, unknown categories, status thresholds, advisory metadata, ordering, duplicates and LF/CRLF bytes.
  • Layer B exports immutable producer source, runs two saved-fixture snapshots, diff and adapt, then invokes the consumer twice in separate output directories. It checks seven golden content contracts, exact and semantic JSONL identity, four schema validations, structured stability and six PNG smoke checks.
  • Architectural decision: ordinary CI includes offline Layer A; Layer B remains a separate reviewer command without a required external checkout or added token authority. The runner never fetches source and refuses existing output directories.

Validation

Fresh Windows CPython 3.12.14 virtual environment; dependencies downloaded from PyPI with no cache/global-package reuse. Both projects installed from source. Resolved pandas 3.0.6, matplotlib 3.11.2, PyYAML 6.0.3 and jsonschema 4.26.0.

  • python -m pip check: passed.
  • python -m pytest tests/test_process_evidence_integration.py -v: 23 passed.
  • python -m pytest tests/test_evidence_pipeline_schemas.py: 8 passed.
  • python -m pytest -q -rs: 274 passed, 3 skipped on Windows (POSIX FIFO and two unavailable directory-symlink cases).
  • python scripts/regenerate_artifacts.py --check: 29 committed artifacts match, six visual outputs regenerated with smoke checks.
  • python scripts/reproduce_process_evidence.py --producer ../process-evidence-producer --output-root ../process-evidence-run: four producer commands + two real consumer runs passed; 7 events, 1 window, 11 numeric features, 4 sources, 7 targets, 0 default alerts.
  • Ruff, Python AST checks, git diff --check, public-content scan and protected-output scope checks passed.
  • Four independent read-only review roles completed. Three disposable-source mutations were rejected (inclusive window end, lost metadata, normalized exact-byte digest). A fourth counterexample exposed an extra-feature false green; exact column-set assertions now reject it, and independent follow-up closed the finding.

Five incremental commits are GitHub Verified, with GPG signatures and noreply DCO trailers:
5cc7d289c3883a6df8c07b4ebaf792036980b51b,
25d17ed11d6884a66e1781334e88c9e197fa2dcf,
78a5de66cf52813dec32b68b0cece51d898ab0e4,
addde0a83173446e8a9c6006a3abd7b2acc360ce,
bbe483eb43ca11bd54c3ffe609e369abc2850980.

Remote validation on bbe483eb43ca11bd54c3ffe609e369abc2850980: Linux/Python 3.11 ordinary CI reports 277 passed; both push and PR runs of CI and Repo Sentinel succeeded (4/4 checks). The dated case-study note is appended so existing reviewed finding positions remain stable; the new guide retains the complete producer SHA beside its repository link. No baseline, suppression, scanner configuration or workflow was changed.

Reviewer instructions and the full command sequence are in the integration guide.

Compatibility

Runtime, default rules, schemas, five demos, producer implementation and release workflows retain their existing behavior. The fixture is synthetic and source-attributed. Comparison-observation timestamps and same-boot process context retain their limited meaning; this suite makes no detection-efficacy, authenticated-provenance or production-readiness claim. The historical case's private receipts remain separate.

Main risk: dependency resolution can drift; Layer B is an explicit reviewer command. PNG checks establish valid images rather than cross-platform byte identity. Rollback removes the added fixture/tests/runner/guide, dated case-study link and scoped LF attribute; no data or schema migration is needed.

Keep this PR Draft. This task authorizes publication for review only; no merge, auto-merge, tag or release.

Signed-off-by: stacknil <219124580+stacknil@users.noreply.github.com>
Signed-off-by: stacknil <219124580+stacknil@users.noreply.github.com>
Signed-off-by: stacknil <219124580+stacknil@users.noreply.github.com>
Signed-off-by: stacknil <219124580+stacknil@users.noreply.github.com>
Signed-off-by: stacknil <219124580+stacknil@users.noreply.github.com>
@stacknil
stacknil marked this pull request as ready for review October 20, 2026 17:08
@stacknil
stacknil merged commit ee7ed44 into main Oct 20, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant