Skip to content

Model packs (<lang>/ext, <lang>/ext-library-sources) silently apply zero models when the consumer's CodeQL CLI doesn't bundle the exact extensionTargets version #206

Closed
#212
Closed
Model packs (<lang>/ext, <lang>/ext-library-sources) silently apply zero models when the consumer's CodeQL CLI doesn't bundle the exact extensionTargets version#206
#212
Labels
bugSomething isn't working

Description

@felickz
felickz
opened on Aug 3, 2026

Summary

Every published model/extension pack in this repo (csharp/ext, csharp/ext-library-sources, go/ext, java/ext, java/ext-library-sources, python/ext) pins extensionTargets to a single exact codeql/<lang>-all version, e.g.:

# java/ext/qlpack.yml
extensionTargets:
  codeql/java-all: '9.2.1'

Per the CodeQL model pack docs, a model pack's data extensions are only injected into an analysis if the codeql/<lang>-all version actually resolved for that analysis satisfies extensionTargets. I've now empirically confirmed that when it doesn't, our sink/source/summary models are silently skipped — no error, no warning that mentions the mismatch — and the analysis just quietly loses that coverage.

This affects any consumer whose CodeQL CLI doesn't happen to bundle exactly the pinned version — not just contributors testing locally.

Reproduction (100% confirmed, not just doc-reading)

Environment: local CLI 2.25.6 (bundles codeql/java-all 9.1.2), repo pinned to CLI 2.26.1 (.codeqlversion), java/ext/qlpack.yml pins extensionTargets: codeql/java-all: '9.2.1'.

  1. codeql pack install java/test (no flags, exactly what ci.yml's "Install Dependencies" step runs):

    Nothing downloaded.
WARNING: Pack 'codeql/java-all' was found via '--additional-packs'. If the generated
'codeql-pack.lock.yml' file is committed to source control, other users will also have
to use '--additional-packs' when compiling or running queries.
  (C:\Utils\codeql\qlpacks\codeql\java-all\9.1.2\qlpack.yml:1,1-1)
...
Nothing to install.

    The CLI does not attempt to fetch the pinned 9.2.1 from the registry — it silently substitutes its own bundled 9.1.2 via the implicit "found via --additional-packs" path, with only a generic reproducibility warning (nothing that says "this doesn't satisfy the pinned version").

  2. codeql test run java/test/security/CWE-089/spring-r2dbc --failing-exitcode=122 --verbosity=progress --ram=2048 --format=json (the exact command ci.yml's "Test Queries" step runs) — the DatabaseClientSqlInjection.ql test (added in Java: model Spring R2DBC DatabaseClient and io.r2dbc.spi as SQL injection sinks #204) fails silently: 0 actual results vs. 6 expected, FAILED(RESULT), no compile error, no diagnostic about codeql/java-all or extensionTargets at all.

  3. Isolated the fix: temporarily changed java/ext/qlpack.yml's extensionTargets from the exact pin '9.2.1' to a range, '>=9.1.0 <=9.2.1', and re-ran step 2 with the same CLI (2.25.6/java-all 9.1.2, nothing else changed) — all 6 expected results now pass. This proves:

    • The MaD row schema for sinkModel/summaryModel is compatible across 9.1.2–9.2.1 (no breaking schema change in that span).
    • A version range instead of an exact pin is a viable, low-risk fix for this specific gap.

(Reverted both experimental edits after confirming — no code changes are included in this issue.)

Root cause (two layers)

  1. CLI dependency resolution silently prefers its own bundled codeql/<lang>-all over fetching a different pinned version from the registry, when the bundled one is found via the CLI's implicit --additional-packs-style search path — with no check that the bundled version actually satisfies the declared dependency/lock-file version, and only a generic "you'll need --additional-packs too" warning (not a version-mismatch warning). This appears to be true in both directions (older CLI + newer pin, or newer CLI + older pin) since resolution isn't semver-gated at that layer.
  2. extensionTargets then silently fails to match whatever codeql/<lang>-all version actually got resolved (per (1)), so the model pack's dataExtensions are never injected into the analysis — with no error/warning at all from this layer either.

Net effect: any consumer of a published <lang>/ext or <lang>/ext-library-sources pack — not just local contributors — whose CodeQL CLI doesn't bundle a codeql/<lang>-all inside our pinned extensionTargets range gets zero of that pack's models applied, silently, with no signal anything went wrong.

Scope

Every language's ext/ext-library-sources pack pins an exact extensionTargets version today, all set by .github/scripts/pin-codeql-library-versions.sh on every .codeqlversion bump:

Pack Current exact pin
csharp/ext, csharp/ext-library-sources codeql/csharp-all: '7.1.0'
go/ext codeql/go-all: '7.2.1'
java/ext, java/ext-library-sources codeql/java-all: '9.2.1'
python/ext codeql/python-all: '7.2.1'

So this isn't specific to the Spring R2DBC models (#204) — it's a systemic, repo-wide characteristic of how every model pack we publish is pinned.

What we can do today (partial, undocumented workaround)

Historical pack versions remain published on GHCR (e.g. githubsecuritylab/codeql-java-extensions has 13 tags going back ~3 years), and each version's OCI manifest carries a com.github.codeql.cli.version annotation recording which CLI built it — so a consumer could in principle pin their own dependency to an older, compatible release matching their CLI. But: (a) nothing documents this mapping anywhere today (a user would need to inspect GHCR manifests directly), and (b) it doesn't fully solve determinism on its own, since layer (1) above can still substitute an unexpected bundled version underneath even a correctly-pinned extensions pack.

Options to consider (needs a maintainer decision, not solo-fixing this)

  1. Broaden extensionTargets to a version range (e.g. >=X.Y.0 or a wider bound) instead of re-pinning to the exact bundle version on every CLI bump — confirmed technically viable above, but needs a decision on how wide is safe (risk: if the MaD schema ever changes within the chosen range, that's a real compile break, not just a silent skip) and whether pin-codeql-library-versions.sh's "always overwrite to exact bundle version" approach should change for extensionTargets specifically (still fine for dependencies:, which resolves/locks correctly).
  2. Document the CLI-version ↔ pack-version compatibility mapping (e.g. a table sourced from com.github.codeql.cli.version manifest annotations) so consumers on older CLIs know which pack version to pin — doesn't fix the silent-failure mode, just makes the existing workaround discoverable.
  3. File an upstream CodeQL CLI issue about layer (1) — packs found via the implicit bundled/--additional-packs path aren't checked against the declared dependency version at all, with only a generic (not version-specific) warning. Searched github/codeql issues; didn't find an existing report of this exact behavior.
  4. Some combination of the above.

Flagging this for a maintainer decision rather than picking one unilaterally, per this repo's convention that CLI-compatibility trade-offs are a judgment call, not something to resolve solo.

Activity

  1. felickz
    added
    bugSomething isn't working
    on Aug 3, 2026

  2. felickz commented on Aug 3, 2026

    @felickz
    felickz
    on Aug 3, 2026
    ContributorAuthor

    Update: reproduced with the real, currently-shipping CLI — this is a live break today, not a future risk

    Just installed the actual CodeQL CLI 2.26.2 (released today, one patch after our repo's pinned 2.26.1 in .codeqlversion) and ran our real java/ext pack against it, unmodified:

    $ codeql --version
CodeQL command-line toolchain release 2.26.2.

$ codeql resolve qlpacks
codeql/java-all is found in 3 same-priority locations...
   C:\Utils\codeql\qlpacks\codeql\java-all\9.2.2
   ...

$ cat java/ext/qlpack.yml
extensionTargets:
  codeql/java-all: '9.2.1'   # <-- does NOT satisfy 9.2.2

$ codeql test run java/test/security/CWE-089/spring-r2dbc
0 tests passed; 1 tests failed:
  FAILED: .../DatabaseClientSqlInjection.ql
  (0 actual results vs. 6 expected — silent, no compile error, no version-mismatch warning)

    This is not a synthetic/edited-pin repro — it's our actual, currently-published java/ext/qlpack.yml (extensionTargets: codeql/java-all: '9.2.1'), run against the actual current CodeQL CLI release (2.26.2, bundling java-all 9.2.2). CI itself is unaffected right now only because ci.yml happens to pin .codeqlversion: 2.26.1 (bundles java-all 9.2.1, an exact match) — but any consumer who has already upgraded to 2.26.2 (a routine patch upgrade, released the same day I tested this) gets zero of our Spring R2DBC SQL-injection models applied, silently, right now.

    This confirms the issue isn't a hypothetical edge case that might bite someone eventually — a single patch-level CLI bump on the consumer's side, with no corresponding action from us, was enough to trigger it. Given how easily/quickly this reproduced with an unmodified, currently-published pack, I think this raises the urgency of picking one of the four options in the issue description — happy to implement whichever the maintainers prefer (my empirical testing above showed option 1, widening extensionTargets to a range like >=9.1.0 <=9.2.2, works without any MaD schema break across that span).

  3. felickz commented on Aug 3, 2026

    @felickz
    felickz
    on Aug 3, 2026
    ContributorAuthor

    Re-verified the range fix directly on 2.26.2 (not just extrapolated from the earlier 2.25.6 test): temporarily set java/ext/qlpack.yml's extensionTargets to codeql/java-all: '>=9.1.0 <=9.2.2', re-ran the same test under CLI 2.26.2 — DatabaseClientSqlInjection.ql now passes (1/1 tests passed), then reverted the temporary edit (confirmed clean git status, no changes left behind). So the range approach is now confirmed to work at both ends of the span (9.1.2 under CLI 2.25.6, and 9.2.2 under CLI 2.26.2), with the same unmodified pack/model rows in between.

  4. felickz commented on Aug 3, 2026

    @felickz
    felickz
    on Aug 3, 2026
    ContributorAuthor

    Answered a follow-up question directly: does it matter if the pack is a genuine GHCR-registry install vs. local --search-path?

    Short answer: no — verified this empirically too, with a fully genuine registry install, not just local disk.

    Warning

    The pack is not merely ignored silently — CodeQL's own compiler logs this, buried in the build output:

    WARNING: Extension pack 'githubsecuritylab/codeql-java-extensions' is unused. (.../githubsecuritylab/codeql-java-extensions/0.7.2/qlpack.yml:1,1-1)

    This is what actually happens when the extensionTargets constraint doesn't match the CLI's resolved codeql/java-all version: the entire pack is discarded at pack-resolution time. It's just a low-severity WARNING, not a build failure — the job still goes green, there's no SARIF-level indication, and nothing surfaces this to a typical consumer unless they know to grep build logs for "is unused".

    Once extensions (java) finished publishing githubsecuritylab/codeql-java-extensions@0.7.2 to GHCR (via the in-flight patch-release run), I:

    1. codeql pack download githubsecuritylab/codeql-java-extensions@0.7.2 — real network pull into ~/.codeql/packages (confirmed: Installed fresh githubsecuritylab/codeql-java-extensions@0.7.2 (library)).
    2. Temporarily moved the local java/ext directory entirely out of the repo tree (not just renamed — CodeQL discovers packs by qlpack.yml name, not folder name, so a rename alone doesn't hide it from --search-path).
    3. Ran codeql pack install --additional-packs .. in java/test — this genuinely could not find githubsecuritylab/codeql-java-extensions locally anymore, fell through to the registry, and locked it for real: 
      # codeql-pack.lock.yml
dependencies:
  githubsecuritylab/codeql-java-extensions:
    version: 0.7.2
      (codeql/java-all resolved to 9.2.2 via the CLI's own bundle, exactly like a normal consumer with no special config.)
    4. Ran the test suite against this fully genuine, registry-resolved dependency graph (CLI 2.26.2, codeql/java-all@9.2.2 bundled, githubsecuritylab/codeql-java-extensions@0.7.2 from GHCR, pinned extensionTargets: '9.2.1' inside that published pack) — identical silent failure, 0/6 expected results, with the same is unused warning in the compile output.
    5. Restored everything (java/ext back in place, codeql-pack.lock.yml reverted via git checkout), confirmed git status clean, and reran the original local-search-path test to confirm no regression from the experiment.

    Conclusion: the extensionTargets version check is evaluated purely against whatever codeql/<lang>-all the CLI resolves for the analysis (its own bundled version, in the common case) — it doesn't matter at all whether the model pack itself was found via local disk/--search-path or genuinely codeql pack install-ed from GHCR with a real lock-file entry. This closes out the last open "maybe it behaves differently for real consumers" question — it doesn't. A real user who does codeql pack install and picks up our just-published 0.7.2 today, on a completely stock CLI 2.26.2, gets zero Spring R2DBC SQLi coverage, with only the low-visibility is unused compiler warning above as any indication — nothing at the SARIF/results level, and no failed build to flag it.

  5. felickz commented on Aug 3, 2026

    @felickz
    felickz
    on Aug 3, 2026
    ContributorAuthor

    Update: reproduced in a real, live hosted GitHub Actions code scanning run (not just local CLI testing)

    To rule out "maybe this only shows up in local/manual testing," reproduced the same failure via an actual hosted GitHub Actions default-setup code scanning run against a small repo referencing the published pack githubsecuritylab/codeql-java-extensions@0.7.2 directly (via repository-level custom configuration, not a workflow file).

    The runner auto-resolved CodeQL CLI 2.26.2 (bundles codeql/java-all 9.2.2). Pack 0.7.2's extensionTargets is pinned to '9.2.1'. The build log shows CodeQL's own compiler dropping the pack outright:

    WARNING: Extension pack 'githubsecuritylab/codeql-java-extensions' is unused.

    And the resulting code scanning analysis for that run came back with results_count: 0 (tool version 2.26.2) — the known SQL injection finding this pack is meant to catch was not detected, purely because of the version-pin mismatch, not a query bug.

    This confirms the behavior is identical across all three reproduction paths tried so far: local unmodified pin, genuine codeql pack install from GHCR, and now a real hosted GitHub Actions default-setup run. Any consumer running CodeQL 2.26.2 (or any CLI whose bundled codeql/java-all falls outside a pinned patch version) silently gets zero coverage from this pack today.

    Next step: 0.7.3 (re-pinned extensionTargets: '9.2.2' to match CLI 2.26.2) is now published — retesting this same scenario next and will report back with results.

  6. felickz commented on Aug 3, 2026

    @felickz
    felickz
    on Aug 3, 2026
    ContributorAuthor

    Confirmed fixed: retested with 0.7.3 (re-pinned extensionTargets: '9.2.2')

    Re-ran the identical scenario from the previous comment, changing only the pack version from 0.7.2 to the just-published 0.7.3:

    packs:
  java:
    - githubsecuritylab/codeql-java-extensions@0.7.3

    Same hosted GitHub Actions default-setup run, same auto-resolved CodeQL CLI 2.26.2 (codeql/java-all 9.2.2). This time:

    • No Extension pack ... is unused warning anywhere in the build log — the pack is accepted.
    • The code scanning analysis for this run came back with results_count: 1 (previously 0), confirming the known SQL injection finding is now detected.

    Clean before/after pair, same repo and same CLI, only the pack version changed:

    Pack version extensionTargets CLI resolved java-all Pack accepted? Results
    0.7.2 '9.2.1' 9.2.2 No — "is unused" 0
    0.7.3 '9.2.2' 9.2.2 Yes 1

    This closes the loop on reproducing both the break and the fix end-to-end in a real hosted default-setup run (not just local CLI testing). The remaining open question for this issue is still the structural one: whether to keep doing exact-version re-pins on every CLI bump (as done here) or move to a range-based extensionTargets constraint so consumers aren't silently dropped between releases.

  7. felickz
    added a commit that references this issue on Aug 4, 2026

    fix: use unconstrained extensionTargets ('*') for all languages, not …

    f14d2b9
  8. felickz
    closed this as completedin #212on Aug 4, 2026

  9. felickz commented on Aug 4, 2026

    @felickz
    felickz
    on Aug 4, 2026
    ContributorAuthor

    Final fix landed & retested end-to-end: extensionTargets broadened to '*' for all four languages (PR #212), released in 0.7.4

    Landed the structural fix flagged as the open question in the previous comment: extensionTargets in every language's ext pack (csharp, go, java, python) is now pinned to '*' instead of an exact/narrow codeql/<lang>-all version. This means future CLI bumps no longer risk silently dropping the whole pack for any consumer whose CLI doesn't happen to match the exact pin — see #212. Published in codeql-java-extensions@0.7.4 (and the equivalent releases for the other three languages).

    Retested the identical repro scenario from this issue against 0.7.4, this time including the case this issue is actually about — a genuinely mismatched/older CLI — not just the "matches-the-current-pin" case:

    CLI (bundled java-all) Pack Result
    2.25.6 (9.1.2) 0.7.3 (exact pin '9.2.2') WARNING: Extension pack 'githubsecuritylab/codeql-java-extensions' is unused. → 0 results (same silent-drop failure documented above)
    2.25.6 (9.1.2) 0.7.4 (extensionTargets: '*') No warning, pack applied → 1 result (SQL injection correctly detected)
    2.26.2 (9.2.2, matches current .codeqlversion) 0.7.4 No warning, pack applied → 1 result

    Same database, same query, only the pack version changes between the first two rows — a clean before/after pair proving the fix (not just the exact-repin band-aid from 0.7.3) resolves the underlying issue for CLIs that don't match whatever the current pin happens to be.

    Also reconfirmed in a live, hosted GitHub Actions default-setup run using 0.7.4: pack installed cleanly with no "is unused" warning, and the finding was reported in code scanning as expected.

    This closes the loop: the original FN is fixed, and the fix has now been validated against a genuinely stale/mismatched CLI (not only the current one), a matching CLI, and a real hosted CI run. Going forward, CLI bumps no longer require re-pinning extensionTargets to avoid this failure mode.

  10. pull
    added a commit that references this issue on Aug 4, 2026

    Fix GitHubSecurityLab#206: use fully unconstrained extensionTargets (…

    a489ca2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions