Skip to content

SDK Synthetic red 60/60 consecutive runs - demo contract test pins free_commodities == 9, live API serves 37 #80

Description

@karlwaldman

Confirmed-live, verified 2026-08-10 15:37 UTC against run 31404558326.

tests/integration/test_demo_contract.py:107 (and 2 sibling assertions) pin len(meta["free_commodities"]) == 9. The demo API has intentionally served 37 free codes since api#5207 merged 2026-07-28 (app/controllers/v1/demo_controller.rb:39 FREE_COMMODITIES expansion - gas hubs/JKM/futures/DI still gated, so this is a stale pin, not a paywall leak).

Every Scheduled SDK Synthetic run since has failed: 60 of the last 60 hourly runs red, nobody acting - the monitor now cries wolf hourly and would mask a real SDK regression.

Fix (3 lines): assert the free list is a superset of the original 9 core codes and >= 9, mirroring the un-pinned catalog-count pattern already in the same file.

Same bug class the file itself warns about two lines above the pin. Labeled from measured impact: the synthetic is our only continuous SDK-against-production check and it has been dark 2.5+ days.

🤖 Generated with Claude Code

Activity

  1. karlwaldman commented on Aug 11, 2026

    @karlwaldman
    MemberAuthor

    Fix prepared — exact red/green receipt

    Draft PR #81 fixes this at 4d191da: #81

    The live red phase reproduced 3 failures / 4 tests: 34 current rows and 37 configured demo codes were compared with stale exact-nine assertions. The corrected contract preserves the original nine-code compatibility floor, requires usable current rows for that core, permits deliberate catalogue growth, and checks the advertised free list for core coverage and duplicates.

    Green proof: live production synthetic 4/4, mocked DemoResource 5/5, Ruff clean, Black clean, diff check clean. No package release, deploy, API mutation, or customer action occurred; keep this issue open until the PR merges and the scheduled synthetic is observed green.

  2. karlwaldman commented on Aug 11, 2026

    @karlwaldman
    MemberAuthor

    Final exact-head receipt

    Exact head: 4d191da09c6dcd4e2e2074f5be28bafdfb6104c5.

    • GitHub: draft, MERGEABLE/CLEAN
    • hosted live API test: green
    • hosted Python 3.8, 3.9, 3.10, 3.11, and 3.12 matrices: all green
    • local red: 3 failures / 4 live tests on current main
    • local green: 4/4 live plus 5/5 mocked DemoResource tests
    • Ruff, Black, and diff check: clean

    No release or deploy occurred. Observe at least one scheduled synthetic run after merge before closing #79/#80 or treating SDK health as promotable.

  3. karlwaldman commented on Aug 11, 2026

    @karlwaldman
    MemberAuthor

    Resolved and production-verified.

    • Fix: PR Fix live demo catalogue synthetic drift #81 merged as fdcd39e6ac0711e0aa1f874a7e127c9dc791acc0. The contract now preserves the original core-code compatibility floor, accepts deliberate catalogue growth, validates every returned core row, and propagates request/transport/OS failures instead of skipping.
    • Post-merge synthetic: run 31487552692 at release head 7f2338c9ba3e73810d868b898da283f983f06f60 is green. Its always-on keyless demo smoke, required monitor credential, bounded authenticated customer path, and 30-day receipt upload all passed: https://github.com/OilpriceAPI/python-sdk/actions/runs/31487552692
    • Released artifact: oilpriceapi==1.12.1, tag/release at the same exact head. Trusted publisher run 31487832170 passed tests, Ruff, dependency audit, exact-wheel install/import, manifest attachment, and PyPI upload: https://github.com/OilpriceAPI/python-sdk/actions/runs/31487832170
    • Public registry verification: a fresh virtual environment installed exactly oilpriceapi==1.12.1, pip check passed, and live keyless plus authenticated Brent/WTI checks returned finite values with source timestamps. Wheel SHA-256: 159308247cc36b8a1233c6b190b736bf2b8336ac180759913c5fdd2c70e60afa.

    This and #79 describe the same stale exact-count incident; no separate work remains.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions