Repository navigation
ci(docker): publish a sortable main-<commit UTC>-<sha7> relay tag - #8240
Conversation
Every main push already publishes :sha-<7> and :main. :sha-<7> carries no order, so a consumer picking "newest main build" has to fetch metadata for every tag. Kargo's NewestBuild strategy does that against ~1,700 sha-* tags (4,800 total) on every warehouse discovery cycle. Add :main-<YYYYMMDDHHmmss>-<7> (and the debug- variant via the merge job's flavor prefix) to the same manifest on push to main. The UTC commit time prefix makes the tag sort by build order, so a Lexical selector can pick the newest from the tag list alone. It uses the commit date, not the run date, so rerunning a commit re-pushes the same tag. Existing tags are unchanged. Signed-off-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Kargo's buzz-image warehouse selects Lexical over the main-<UTC>-<sha7> tag; a drifted format would stop Freight discovery silently. Also note the tag form in the chart README's Datadog version section. Signed-off-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
🔐 Codex Security Review
Review SummaryOverall Risk: NONE
FindingsNo concrete security, correctness, or reliability findings were identified. Notes
Generated by Codex Security Review | |
wpfleger96
left a comment
There was a problem hiding this comment.
🤖 Thanks for this. Checked against the pinned docker/metadata-action v6.1.0 source:
type=rawvalues go through the same handlebars pass as everything else, so{{commit_date}}and{{sha}}resolve.commit_datedefaults totz=UTC, and{{sha}}is the 7-char short sha, same as:sha-<7>.- On a push, the commit date comes from the event payload (
commits[].timestamp/head_commit), so reruns re-push the same tag. - The merge job's
prefix=${{ matrix.tag_prefix }}flavor yieldsdebug-main-…, and the anchored^main-regex excludes it. - The last 300 commits on main have author time equal to committer time, and both increase monotonically. Lexical order matches build order.
Nit, optional: the README edit leaves one long line inside an otherwise wrapped paragraph. It renders the same either way.
## Summary This PR makes two changes to the push gateway image job (`push-gateway-merge` in `docker.yml`). Together they let staging auto-promote the gateway the same way it already auto-promotes the relay: 1. **Publish only after same-SHA CI qualification.** `push-gateway-merge` now needs `qualify` as well as `push-gateway-build`. This is the same gate the relay's `merge` job already uses: a successful `ci.yml` push run on the exact source SHA. Before this change, the gateway image was published even when main CI was red. That was safe while staging pinned the gateway by hand. It isn't safe once the newest main image deploys automatically. 2. **Publish the sortable `:main-<YYYYMMDDHHmmss>-<sha7>` tag on main pushes.** It's the same `type=raw` line the relay matrix gained in #8240. It uses the same `enable=` guard, so it applies only to `push` on `refs/heads/main`. A Lexical selector on `^main-\d{14}-[0-9a-f]{7}$` can then pick the newest gateway build from the tag list alone. `:sha-<7>` has no ordering. `:main`, `:sha-<7>`, and the release semver tags don't change. The gateway has no debug variant, so no `debug-main-*` tag is published. **Tradeoff:** a gateway image now appears only after `qualify` finishes, which takes as long as main CI. If main CI fails, no gateway image is published for that commit, and that includes `:main` and `:sha-<7>`. The relay already behaves this way, and it's the behavior we want for anything that auto-deploys. **What got simpler:** the relay and gateway images now share the same publication gate and the same tag set. Downstream tooling no longer needs a special case for the gateway's unqualified, unordered tags. **What stays different:** the relay's `merge` job also attests the `deployment-eligibility/v1` predicate (`docs/deployment-identity.md`); the gateway publishes no such attestation. That's intentional for now: the staging promotion guard selects the gateway by package and `main-*` tag and does not verify attestations, so the gateway warehouse must not be wired to require one. If promotion ever starts verifying eligibility, the gateway needs the same attestation first. ### Related issue N/A (staging rollout tooling). The downstream promotion-guard and warehouse changes land separately. The warehouse switches on only after this tag has been published. ### Testing - `scripts/test-relay-image-eligibility-workflow.sh` now pins both properties of the `push-gateway-merge` job (it runs in CI under "Relay image eligibility contract"). Both checks match whole active lines, so a commented-out copy can't satisfy them. - At `9b8c49600` it passes. - With the live `needs` line replaced by `needs: [push-gateway-build]` and a commented-out `# needs: [push-gateway-build, qualify]` left behind, it fails. - With the gateway `main-*` line commented out, it fails. - With `needs` reverted to `push-gateway-build`, it fails with `push gateway image must publish only after same-SHA CI qualification`. - With the `main-*` tag line removed, it fails with `push gateway image must publish the sortable main-<commit UTC>-<7> tag on main pushes`. - `scripts/test-release-ref-contract.sh` now counts the sortable `main-*` tag line on all three image legs (relay, relay-debug, gateway) instead of two. It passes at `9b8c49600`. - `bin/actionlint .github/workflows/docker.yml` is clean at `f6c03b6b5`. - PR runs don't push images. The first real `main-*` gateway tag appears on the first main push after merge, and I'll check the registry for it on that manifest. --------- Signed-off-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Summary
Each push to main adds one more tag to the relay image manifest that
docker.ymlalready publishes::main-<YYYYMMDDHHmmss>-<sha7>(UTC commit time):debug-main-<YYYYMMDDHHmmss>-<sha7>on the debug manifest:main,:sha-<7>, and the release tags don't change.Why:
:sha-<7>has no ordering, so a deploy tool that wants "the newest main build" has to fetch metadata for every tag before it can sort. Our staging promotion warehouse (Kargo,NewestBuild) does that on every discovery cycle, against about 1,700sha-*tags (4,800 total) inghcr.io/block/buzz. That's slow, it hits registry rate limits, and one failed fetch aborts the whole cycle. The timestamp prefix makes the tag sort lexically in build order. ALexicalselector with^main-\d{14}-[0-9a-f]{7}$can then pick the newest build from the tag list alone and fetch only the tags it keeps.Details, checked against the pinned
docker/metadata-actionv6.1.0 (80c7e94d):{{sha}}is the 7-char short sha (defaultShortShaLength = 7), the same value as:sha-<7>.{{commit_date}}formats in UTC by default. On a push it uses the event'scommits[].timestamp, so rerunning a commit produces the same tag.type=rawinherits the flavor prefix, so the merge job's debug leg emitsdebug-main-…. That's the same mechanism that producesdebug-maintoday.enable=limits the tag topushonrefs/heads/main. Release tags, rescue dispatches, and PRs never get it.On main's last 40 commits, author time and committer time are the same instant, and both increase monotonically. A commit pushed with a backdated timestamp would sort below newer builds and just wouldn't be auto-picked; the next push supersedes it.
What got simpler: this adds one tag line in each of the two places that already mirror the tag matrix, plus header docs. In return, downstream tooling no longer needs per-tag metadata fetches to find the newest build.
Related issue
N/A (staging rollout tooling). The downstream warehouse and tag-guard changes land separately and don't switch over until this tag has been published.
Testing
scripts/test-relay-image-eligibility-workflow.shandscripts/test-release-ref-contract.shpass at9ba1d8b7f.push: false), so the first real tag appears on the first main push after merge. I'll check the registry formain-<14 digits>-<sha7>anddebug-main-…on that manifest.