Skip to content

Antalya 26.8: Iceberg: reuse the Puffin object metadata across deletion-vector reads - #2665

Merged
zvonand merged 1 commit into
antalya-26.8from
feature/antalya-26.8/pr-2419
Oct 25, 2026
Merged

zvonand merged 1 commit into
antalya-26.8from
feature/antalya-26.8/pr-2419

Conversation

@zvonand

@zvonand zvonand commented Oct 24, 2026

Copy link
Copy Markdown
Member

Changelog category (leave one):

  • Performance Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Reduce number of Puffin file HEAD requests (#2419 by @UnamedRus).

CI/CD Options

Exclude tests:

  • Fast test
  • Integration Tests
  • Stateless tests
  • Stateful tests
  • Performance tests
  • All with ASAN
  • All with TSAN
  • All with MSAN
  • All with UBSAN
  • All with Coverage
  • All with Aarch64
  • All Regression
  • Disable CI Cache

Regression jobs to run:

  • Fast suites (mostly <1h)
  • Aggregate Functions (2h)
  • Alter (1.5h)
  • Benchmark (30m)
  • ClickHouse Keeper (1h)
  • Iceberg (2h)
  • LDAP (1h)
  • Parquet (1.5h)
  • RBAC (1.5h)
  • SSL Server (1h)
  • S3 (2h)
  • S3 Export (2h)
  • Swarms (30m)
  • Tiered Storage (2h)

Cherry-picked from #2419.


Documentation entry for user-facing changes

...

Iceberg: reuse the Puffin object metadata across deletion-vector reads
@zvonand zvonand added releasy Created/managed by RelEasy antalya-26.8 Session label (releasy session config) forwardport This is a frontport of code that existed in previous Antalya versions labels Oct 24, 2026
@github-actions

Copy link
Copy Markdown

Workflow [PR], commit [8546561]

@zvonand zvonand mentioned this pull request Oct 24, 2026
58 of 59 tasks
@zvonand

zvonand commented Oct 25, 2026

Copy link
Copy Markdown
Member Author

RelEasy analyze-fails — UNRELATED

run completed at 2026-10-25T09:47:16Z

  • Head SHA: 85465612b3 (feature/antalya-26.8/pr-2419)
  • Failed CI checks: 2
  • Tests considered: 4 across 2 CI shard(s)
  • Baseline (cdbcf6d5f8, 2026-10-23T21:17:55Z, before this PR): 4 of 4 failure(s) were already red there
  • Outcomes: 0 done · 0 partial · 2 unrelated · 0 unresolved
  • Second opinion: no shard was in doubt (nothing committed, and the evidence backs each verdict).
  • Commits added by AI: 0 (—)
  • Anthropic cost: —
  • Checks covered by another shard (same failures, investigated once):
    • Regression release swarms: 3 of 4 failure(s) duplicate Regression aarch64 swarms; the remaining 1 form this shard.

Per-shard outcomes

⏭️ UNRELATED — Regression aarch64 swarms

3 failed test(s) considered — cost $0.2736
full report

AI narration

This PR didn't cause any of the 3 failures in this shard. All three were already failing on antalya-26.8 at baseline commit cdbcf6d5f8 (2026-10-23), which doesn't contain this PR's change. I didn't change any code, run a build, or reproduce anything.

I couldn't do the spot-check against the reports: fetching fails.log.txt from S3 needed approval and was denied. The baseline verdict therefore rests on the annotations that came with the task, plus a check of the diff. The PR being tested (#2419, puffin-dv-metadata-threading) threads Iceberg puffin deletion-vector metadata. That has nothing to do with swarm cluster discovery or how nodes behave when a CPU is overloaded.

  • [unrelated] /swarms/feature/cluster discovery/check cluster discovery with wrong cluster name — already failing at baseline cdbcf6d5f8 (Regression release swarms).
  • [unrelated] /swarms/feature/node failure/cpu overload — already failing at baseline cdbcf6d5f8; also fails in the Regression release swarms shard.
  • [unrelated] /swarms/feature/node failure/cpu overload all swarm nodes — already failing at baseline cdbcf6d5f8; also fails in the Regression release swarms shard.

git status --porcelain is not empty. It lists three untracked .releasy-discover-deps-*/ directories, which were already there when I started. I didn't create them and left them in place.

UNRELATED
This PR didn't cause any of the 3 failures in this shard. All three were already failing on antalya-26.8 at baseline commit cdbcf6d5f8 (2026-10-23), which doesn't contain this PR's change. I didn't change any code, run a build, or reproduce anything.

I couldn't do the spot-check against the reports: fetching fails.log.txt from S3 needed approval and was denied. The baseline verdict therefore rests on the annotations that came with the task, plus a check of the diff. The PR being tested (#2419, puffin-dv-metadata-threading) threads Iceberg puffin deletion-vector metadata. That has nothing to do with swarm cluster discovery or how nodes behave when a CPU is overloaded.

  • [unrelated] /swarms/feature/cluster discovery/check cluster discovery with wrong cluster name — already failing at baseline cdbcf6d5f8 (Regression release swarms).
  • [unrelated] /swarms/feature/node failure/cpu overload — already failing at baseline cdbcf6d5f8; also fails in the Regression release swarms shard.
  • [unrelated] /swarms/feature/node failure/cpu overload all swarm nodes — already failing at baseline cdbcf6d5f8; also fails in the Regression release swarms shard.

git status --porcelain is not empty. It lists three untracked .releasy-discover-deps-*/ directories, which were already there when I started. I didn't create them and left them in place.

UNRELATED

⏭️ UNRELATED — Regression release swarms

1 failed test(s) considered — cost $0.2289
full report

AI narration

The only failure in this shard was already failing before this PR, so I changed no code and committed nothing.

Triage:

  • Baseline: the failing test was already red on antalya-26.8 at commit cdbcf6d5f8 (2026-10-23T21:17:55Z), in the same Regression release swarms shard, before this PR's diff existed. That rules this PR out as the cause.
  • Spot-check not done: I tried to download this run's fails.log.txt to compare it against the baseline, but the download needed approval and didn't run. I also didn't open the baseline report, so I'm relying on the baseline verdict I was given.
  • Diff: the PR is a large rebase-style diff (CI workflows, contrib bumps, Poco SSL and more). I didn't find anything in it specific to swarm node-restart handling, but I only looked at the file list.
  • Not done: I didn't build or run the regression suite, because the baseline already settles it.

git status --porcelain isn't empty, but nothing in it is mine. It lists three untracked .releasy-discover-deps-*/ directories (5b7950, 7b21af, ff3f2d) that were already there when the session started. I left them alone.

Failing tests:

  • /swarms/feature/node failure/check restart swarm node — [unrelated]: already failing at baseline commit cdbcf6d5f8 on antalya-26.8, without this PR's changes. The CI excerpt only says AssertionError.

UNRELATED
The only failure in this shard was already failing before this PR, so I changed no code and committed nothing.

Triage:

  • Baseline: the failing test was already red on antalya-26.8 at commit cdbcf6d5f8 (2026-10-23T21:17:55Z), in the same Regression release swarms shard, before this PR's diff existed. That rules this PR out as the cause.
  • Spot-check not done: I tried to download this run's fails.log.txt to compare it against the baseline, but the download needed approval and didn't run. I also didn't open the baseline report, so I'm relying on the baseline verdict I was given.
  • Diff: the PR is a large rebase-style diff (CI workflows, contrib bumps, Poco SSL and more). I didn't find anything in it specific to swarm node-restart handling, but I only looked at the file list.
  • Not done: I didn't build or run the regression suite, because the baseline already settles it.

git status --porcelain isn't empty, but nothing in it is mine. It lists three untracked .releasy-discover-deps-*/ directories (5b7950, 7b21af, ff3f2d) that were already there when the session started. I left them alone.

Failing tests:

  • /swarms/feature/node failure/check restart swarm node — [unrelated]: already failing at baseline commit cdbcf6d5f8 on antalya-26.8, without this PR's changes. The CI excerpt only says AssertionError.

UNRELATED


🤖 Posted automatically by releasy analyze-fails. Re-run the command to refresh.

@zvonand

zvonand commented Oct 25, 2026

Copy link
Copy Markdown
Member Author

@blau-ai

@blau-ai

blau-ai commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator

CI triage for #2665

Verdict: 1 failing check family, 0 PR-caused. The only red is the Regression swarms suite (aarch64 + release). Every build, Fast test, stateless/stateful, integration, stress, AST fuzzer, SQLLogic, compatibility and install-packages check is green. The swarm failures are pre-existing / flaky on antalya-26.8 and unrelated to this change.


What's failing

Regression aarch64 swarms / Regression release swarms (+ the matching RegressionTests*/Swarms/swarms GH jobs). Failing leaf scenarios:

Scenario arm x86
cluster discovery/check cluster discovery with wrong cluster name ✗ ✗
node failure/cpu overload ✗ ✗
node failure/cpu overload all swarm nodes ✗ ✗
node failure/check restart swarm node — ✗

All are AssertionError on query exit codes inside swarm cluster-management scenarios (cluster discovery, node restart under load, CPU overload) from Altinity/clickhouse-regression — e.g. assert result.exitcode in expected_exitcodes in swarms/tests/node_failure.py:108.

Why this is not caused by the PR

  • Scope mismatch. This PR changes exactly one file — src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergDeletionVector.cpp — threading an optional ObjectMetadata into the Puffin read path to avoid redundant S3 HEADs on deletion-vector loads. There is no path by which that touches swarm cluster discovery, node-failure handling, or CPU-overload behavior. The Iceberg-related swarm scenarios (iceberg fallback…, iceberg engine table fallback) are green/skipped.
  • Same failures on unrelated PRs. The identical scenarios (wrong cluster name, cpu overload, cpu overload all swarm nodes) fail on:
  • Arch-dependent flakiness. 3 fail on aarch64 vs 4 on x86, differing by one scenario (check restart swarm node) — a genuine regression would fail deterministically, not drift by arch/run. These are timing/load-sensitive cluster tests.

Suggested next step

  • No PR-side fix is needed or possible here — the swarm suite is red branch-wide independent of PR content.
  • Safe to re-run the swarm jobs if you want a cleaner board, but don't expect them to reliably go green until the underlying flakiness on antalya-26.8 is addressed (tracked at the suite/infra level, not in this PR). It should not block merge.

PR health check

A small, well-scoped optimization: the Puffin ObjectMetadata is now seeded into RelativePathWithMetadata so createReadBuffer skips its per-buffer getObjectMetadata (S3 HEAD) when the caller already has it — paying one HEAD per deletion-vector load instead of one per buffer, with std::nullopt passed on the paths that don't have the metadata yet (behavior-preserving). All functional CI is green; the change carries no new tests, which is fine for a pure request-count optimization with no observable semantic change.

🤖 Automated triage by @blau-ai. CI reports read from the praktika S3 bucket; swarm comparisons against #2661/#2666.

@zvonand
zvonand merged commit f34df52 into antalya-26.8 Oct 25, 2026
323 of 327 checks passed
@zvonand zvonand added the verified Approved for release label Oct 25, 2026
@zvonand zvonand added the port-antalya PRs to be ported to all new Antalya releases label Oct 28, 2026
@blau-ai blau-ai mentioned this pull request Oct 28, 2026
9 of 30 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

antalya-26.8 Session label (releasy session config) forwardport This is a frontport of code that existed in previous Antalya versions port-antalya PRs to be ported to all new Antalya releases releasy Created/managed by RelEasy verified Approved for release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants