Skip to content

Split Stage 12 coordination and the execution contracts under the file-length limit - #933

Merged
anth-volk merged 1 commit into
mainfrom
file-length-split-stage12-coordination
Oct 27, 2026
Merged

anth-volk merged 1 commit into
mainfrom
file-length-split-stage12-coordination

Conversation

@anth-volk

Copy link
Copy Markdown
Contributor

Fixes #932

Same-Repository Draft PR Check

  • This PR is a draft.
  • This PR is from a branch in PolicyEngine/policyengine-sim-api, not a personal fork.

Summary

Fourth of nine stacked PRs; it builds on #930. It splits Stage 12 report coordination and the contract library's Stage 12 execution contracts without changing behaviour, and removes both from the file-length baseline (count 10 to 8). For each file, a script compared every top-level statement of the old module with the new modules: all appear exactly once and unchanged, apart from the renames below and one logger line, in their original order.

stage12_runtime/coordination.py (1,091 lines)

Module Lines Holds
coordination.py 123 coordinate_report, and re-exports of every public name, in __all__
report_coordination.py 626 CoordinatorInvocation, CoordinatorDependencies, the polling constants, and the coordinator class, which runs one claimed report
coordinated_simulations.py 262 the child simulations' records, failures and cancellation
parent_claim.py 167 checking a dispatched report against its parent record, and claiming the parent
  • report_coordination.py is above the 400 lines the split aims for, because the coordinator class itself is 491 lines; dividing a class is not a move, so it stays whole.
  • The child-failure logging keeps the logger policyengine_simulation_executor.stage12_runtime.coordination, which the runtime tests' caplog checks filter on; coordinated_simulations.py names it explicitly, the one statement that isn't verbatim.
  • Renames, each private name another module now uses: _ReportCoordination to ReportCoordination; _cancel_pending_simulations, _cancellation, _child_failure_detail, _child_record (docstring added), _record_child_failure, _simulation_failure_attributes (docstring added), _validate_dispatch_context, _validate_submitted_parent and _claim_parent drop their underscore.

policyengine_simulation_contract/stage12_execution.py (929 lines)

The module becomes a package with the same import path; its __init__.py re-exports every name, including the two UK geography types it re-exported before.

Module Lines Holds
stage12_execution/__init__.py 114 re-exports, in __all__
stage12_execution/base.py 220 the field types, StrictContractModel, and one simulation's execution input with its dataset, population, geography and provenance models
stage12_execution/report_inputs.py 208 the report's simulation pair, its output requirements and the planned output schema
stage12_execution/result_artifacts.py 185 the stored simulation, report and comparison artifacts
stage12_execution/comparison_runs.py 365 the invocation context, the comparison-run statuses, and the parent and child records
  • Why a package, and how ruff sees the models. Ruff decides that a Pydantic model's annotation imports are needed at runtime by reading the model's bases through its own module's imports, and ruff.toml lists policyengine_simulation_contract.stage12_execution.StrictContractModel and .SimulationExecutionInput. A model in a new module that imported its base from base.py would name an unlisted path, so ruff would move its annotation imports under TYPE_CHECKING and sim.quality.runtime-evaluated-models-resolve would fail; listing the new path would change a ruff setting, which this work doesn't. As a package, the models subclass those bases through policyengine_simulation_contract.stage12_execution, the path already listed. __init__.py imports base.py first (its name sorts first), so the bases are bound in the package before the other modules import them; a comment in each module says so.
  • Every public name has the same value, and every model's JSON schema is byte-for-byte the same; the only difference a script found is the classes' module path, for example in the PopulationInput alias's representation.
  • The two # noqa: PLC0414 suppressions on the UK geography re-exports are gone: __all__ now marks them as re-exports.

Tests

  • No test or test count changes: the executor's 1,873 tests pass as before (22 skipped, as before).
  • tests/test_stage12_coordination.py: the fixed_clock fixture patches datetime in coordinated_simulations, parent_claim and report_coordination, the three modules that read the clock, and uuid4 in report_coordination; test_default_output_plan_resolver_measures_its_own_stages patches resolve_report_output_plan in report_coordination, where the coordinator calls it. tests/test_stage12_runtime.py still patches runtime_store in coordination, where coordinate_report reads it.

Environment variables

No change: the moved code reads no environment variable, and no workflow, image or deployment file changes. The Modal images copy the contract package and stage12_runtime whole, and the image smoke's path filter covers both directories.

Testing

  • Executor, full suite: 1,873 passed, 22 skipped.
  • Contract library: 322 passed. Entrypoint: 742 passed. Gateway: 268 passed. Stage 12 persistence: 40 passed. Observability and fastapi libraries: 38 and 14 passed.
  • test_runtime_evaluated_annotations.py in the executor and the entrypoint (the sim.quality.runtime-evaluated-models-resolve guard): passed.
  • Pyright: every package passes except projects/policyengine-apis-integ, which needs the generated clients (./scripts/generate-clients.sh) to install and isn't touched here.
  • Ruff 0.15.22 format and lint: clean.
  • python scripts/run_quality_guards.py and python -m scripts.rules guard --base-ref origin/file-length-split-gateway-endpoints: passed; python -m scripts.rules render leaves no diff.

Only the lowest PR in the stack runs full CI, so this PR's CI complete check fails by design until the PRs below it merge.

🤖 Generated with Claude Code

@anth-volk
anth-volk added this pull request to stack #928 October 27, 2026 13:07
@anth-volk
anth-volk force-pushed the file-length-split-stage12-coordination branch from 45c5ed8 to 2057f26 Compare October 27, 2026 13:45
@anth-volk
anth-volk force-pushed the file-length-split-stage12-coordination branch 2 times, most recently from 45c5ed8 to dd13bbf Compare October 27, 2026 14:48
Base automatically changed from file-length-split-gateway-endpoints to main October 27, 2026 16:17
…modules

stage12_runtime/coordination.py (1,091 lines) and the contract's
stage12_execution.py (929 lines) were over the file-length limit. The
code moves unchanged.

coordination.py keeps coordinate_report and re-exports every public
name. The coordinator class, ReportCoordination, its dataclasses and the
polling constants move to report_coordination.py, the child simulations'
records, failures and cancellation to coordinated_simulations.py, and
the parent report's checks and claim to parent_claim.py. Child failures
still log under the coordination module's logger, now named explicitly.
Private names another module uses are public: _ReportCoordination,
_cancel_pending_simulations, _cancellation, _child_failure_detail,
_child_record and _simulation_failure_attributes (both with a new
docstring), _record_child_failure, _validate_dispatch_context,
_validate_submitted_parent and _claim_parent.

stage12_execution becomes a package with the same import path, whose
__init__.py re-exports every name: base.py holds the field types, the
strict contract base and the simulation execution input; report_inputs.py
the report input and output plans; result_artifacts.py the artifact
descriptors and payloads; comparison_runs.py the invocation context and
the comparison-run records. The models in the other modules subclass
StrictContractModel and SimulationExecutionInput through the package
path, which ruff.toml already lists as runtime-evaluated, so ruff keeps
their annotation imports at runtime with no settings change; base.py
sorts first, so the package binds those bases before the other modules
import them. Every model's JSON schema is unchanged.

The coordination tests' fixed clock patches datetime in each of the
three modules that read it, and uuid4 and resolve_report_output_plan in
report_coordination.py, where the coordinator looks them up.

The file-length baseline drops both files (count 10 to 8).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk
anth-volk force-pushed the file-length-split-stage12-coordination branch from dd13bbf to 4afab74 Compare October 27, 2026 16:17
@anth-volk
anth-volk marked this pull request as ready for review October 27, 2026 16:30
@anth-volk
anth-volk requested a review from a team as a code owner October 27, 2026 16:30
@anth-volk
anth-volk merged commit 1156b40 into main Oct 27, 2026
23 checks passed
@anth-volk
anth-volk deleted the file-length-split-stage12-coordination branch October 27, 2026 16:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Split Stage 12 coordination and the execution contracts under the file-length limit

1 participant