[anth-volk]

Fixes #4135

Summary

tests/unit/test_country_spm.py ran 37 of its 63 tests against the installed policyengine-us model, through function-scoped fixtures that reloaded the 26.6 MB SPM forecast and built a new Flask app for every test. On #4039 the file took about 14 of the Test job's 41 minutes.

This PR makes the file a unit test file and keeps two checks against the installed model as integration tests.

• Unit tests (tests/unit/test_country_spm.py). Every case now runs through the household routes and the real PolicyEngineCountry.prepare_calculation, _create_simulation and calculate code, over a stand-in for the country package's Simulation class. A stand-in raises the error the installed model raises for that household, with the model's code and message, or reports the model's receipt shape. The tests then check what the Public API does with it: status, code, message, null cells, receipt and caching. Nothing in the file uses COUNTRIES["us"], load_forecast() or a real Simulation. The longest test takes 0.01 s.
• Integration tests (tests/integration/test_installed_us_model_spm.py). Two tests run the installed model. Session-scoped fixtures build the Flask app and its calculation service once.
• Where they run. make test lists the integration file next to tests/integration/test_budget_window_in_flight_dedupe.py, the one integration file it already ran. The file needs no database, network or credentials. So it still runs in the pull request Test job and in push.yml's release tests, before the staging deploy, as the real-model checks did. A separate job would save no wall-clock time: on Report which parameters a policy sets that the current model no longer has #4039 the cross-database integration job took 29.8 minutes, longer than the Test job will take without these tests.
• /us/calculate-full. A stand-in test checks the route's own behaviour: it adds every omitted yearly variable, calculates them under the chosen measurement, and returns the receipt. No test calculates every variable of the real model any more.
• Private names. The integration tests call no private names; they use the installed bundle as it is. The unit stand-ins certify a stand-in bundle through spm._current_bundle, spm._selected_forecast and spm.simulation_supports_spm, the same seams tests/unit/routes/test_canonical_spm.py, test_spm_settings.py and test_readiness.py use. No public route can replace the installed bundle.

The two integration tests

Test	What only the installed model shows	Local time	Under coverage run --branch
test_a_chosen_county_yields_a_threshold_and_the_installed_forecasts_receipt	The model prices Los Angeles County with the forecast the bundle pins. Its receipt passes the Public API's validation, names that forecast's sha256 and scenario and the year, and records the installed policyengine-us and spm-calculator versions.	0.08 s	0.43 s
test_a_state_only_household_is_rejected_only_when_it_chose_the_measurement	The docs/canonical-spm.md commitment, end to end. Without spm, the household inherits the county default and gets HTTP 200 with the threshold null. With spm: {"geography_kind": "county"}, the model's missing-county error becomes a 400 with SPM_GEOGRAPHY_REQUIRED and its own text.	0.05 s	0.40 s

Times are each test's call time on an Apple-silicon Mac. Session setup takes 0.01 s: importing the country module already loads the US model, and the US metadata it builds loads the Public API's cached copy of the forecast. Process start-up, mostly importing the country models, takes 17 s without coverage and 87 s under coverage; make test pays that once for the whole run.

The second test checks both halves of the commitment in one test. The inherited half alone cannot show that the model raises a typed SPM input error: the country leaves any failed cell null when nothing was chosen, typed or not. The chosen half shows the typed code.

Where each real-model check went

Original test (cases)	Now
test_real_country_state_only_tax_succeeds_and_receipt_stays_empty (1), test_real_http_tax_only_succeeds_without_geography (1), test_real_http_tax_only_outside_artifact_years_is_lazy (4)	Unit: test_http_tax_only_result_carries_a_receipt_that_measured_no_year
test_real_country_state_only_spm_dependency_requires_geography (2), test_real_http_spm_errors_use_validation_response (5), test_real_country_unknown_county_is_structured (2), test_real_country_unclassified_composition_is_structured (1), test_real_http_unsupported_spm_year_is_structured_only_when_calculated (8)	Unit: test_http_reports_a_chosen_measurements_missing_input_as_a_400, one case per household (no county, county not a FIPS code, county the forecast cannot place, no classified adult, year after the forecast), with and without axes. Integration: the chosen half of the state-only test.
test_real_country_state_only_dependency_is_null_when_nothing_was_chosen (2), test_real_http_missing_primitives_are_null_when_nothing_was_chosen (4)	Unit: test_http_returns_an_inherited_measurements_dependants_as_null, the same households without spm, with and without axes. Integration: the inherited half of the state-only test.
test_real_country_unknown_area_is_structured (1)	Unit: test_http_reports_an_area_the_forecast_lacks_before_any_simulation
test_real_country_explicit_geography_yields_json_receipt (3)	Unit: test_http_explicit_geography_returns_the_threshold_and_its_receipt (national, county, metro), and test_calculate_leaves_the_callers_household_unchanged for its check that the country does not write into the caller's household. Integration: the chosen-county test.
test_real_http_national_result_and_receipt_survive_response_cache (1)	Already a unit test: tests/unit/routes/test_canonical_spm.py::test_http_cache_varies_with_measurement_settings asserts the identical cached national response, receipt included, after one country call.
test_real_http_full_national_calculation_keeps_threshold_and_provenance (1)	Unit: test_http_full_calculation_adds_omitted_variables_under_the_chosen_measurement
test_real_http_stored_national_replay_preserves_threshold_and_cached_receipt (1)	Unit: test_http_stored_household_replays_its_saved_selection_once_then_from_the_cache

Which code the installed model raises for each missing input is policyengine-us's to test. The unit stand-ins use the messages the installed model raised for these households on this branch's lockfile.

Review

• Round 1. One bug: the original explicit-geography test's check that PolicyEngineCountry.calculate leaves the caller's household unchanged had been dropped, and the HTTP tests cannot see it, because the calculation service copies the household first. It is restored as test_calculate_leaves_the_callers_household_unchanged. I confirmed that test fails when the copy in prepare_calculation is removed. Code-quality findings, all fixed:
  • The HTTP stand-ins replaced _create_simulation. They now stand in for the package's Simulation, so the second SPM normalization and the constructor arguments run.
  • The module docstrings claimed that nothing loads the model, and gave the wrong reason for the session fixtures.
  • The testing skill's rule against the real model covered every unit test, which other modules contradict.
  • The state-only integration test relied, without saying so, on the bundle's default geography being a county.
  • The older stand-in tests repeated literals that the module now defines once.
  • One finding is left for a follow-up: certified_bundle and http_client_for repeat the shape of certified and harness in tests/unit/routes/test_canonical_spm.py. Sharing them would change that module's fixtures, which is outside this PR.
• Round 2. Running.

Testing

Run only the tests this PR adds or modifies, each selected by its node ID:

• tests/unit/test_country_spm.py: every test_http_* case (27), test_missing_input_cases_cover_every_spm_input_error_code, test_calculate_leaves_the_callers_household_unchanged, test_country_never_swallows_spm_input_errors_for_a_chosen_measurement (8), test_inherited_measurement_leaves_dependent_variables_unavailable (8), test_a_configuration_failure_is_never_left_as_a_null_cell (6), test_tax_only_result_reads_provenance_without_calculating_spm and test_an_inherited_measurement_still_returns_the_rest_of_the_calculation, by node ID with uv run --frozen --extra dev python -m pytest: all pass, none over 0.01 s. test_country_passes_resolved_spm_and_uses_the_simulations_private_system and test_uncertifiable_country_receipt_does_not_become_an_internal_failure are unchanged and were not run.
• tests/integration/test_installed_us_model_spm.py: both tests by node ID, with and without coverage run --branch: both pass.
• ruff format --check ., ruff check ., uv run --frozen --extra dev mypy and python scripts/run_quality_guards.py: all pass.

No full test suite, whole test file or make test was run.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.10%. Comparing base (e615bf3) to head (7bcdfc6).
⚠️ Report is 19 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4136      +/-   ##
==========================================
- Coverage   88.13%   88.10%   -0.03%     
==========================================
  Files         273      273              
  Lines       16299    16299              
  Branches     2668     2668              
==========================================
- Hits        14365    14361       -4     
- Misses       1268     1270       +2     
- Partials      666      668       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.