Skip to content

Antalya 25.8 Backport of #86933 - RFC: Optimize MarkRanges by using devector over deque - #2638

Merged
zvonand merged 1 commit into
antalya-25.8from
backports/antalya-25.8/86933
Oct 25, 2026
Merged

zvonand merged 1 commit into
antalya-25.8from
backports/antalya-25.8/86933

Conversation

@zvonand

@zvonand zvonand commented Oct 22, 2026

Copy link
Copy Markdown
Member

Backport of ClickHouse#86933

Changelog category (leave one):

  • Performance Improvement

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Improves performance of fast queries with lots of parts in table (by optimizing MarkRanges by using devector over deque) (ClickHouse#86933 by @azat)

RFC: Optimize MarkRanges by using devector over deque
@zvonand zvonand added the backport Backport label Oct 22, 2026
@github-actions

Copy link
Copy Markdown

Workflow [PR], commit [e93b59b]

@zvonand

zvonand commented Oct 24, 2026

Copy link
Copy Markdown
Member Author

Backporter: CI failures at e93b59b

Failed checks:

Unrelated to this PR: The PR only swaps the MarkRanges container from deque to devector; the failures are a Hybrid-table reload problem from an existing test, a known flaky ZooKeeper DROP_RANGE error, and infrastructure jobs.

  • Stress test (amd_msan): not caused by the PR. The server failed to restart because the Hybrid table from the existing test 03644_hybrid_unqualified_table points at a dropped segment table, which is a Hybrid-engine reload problem.
  • Stateless tests (amd_ubsan, parallel): not caused by the PR. The server aborted on the flaky ReplicatedMergeTree error "Part … remains in ZooKeeper after DROP_RANGE", and the tests after that failed because the server had died.
  • SourceUpload and RegressionTestsAarch64 S3 (azure): not caused by the PR. Neither report has details, and both look like infrastructure or external-storage failures.
  • Changes: none. The PR's MarkRanges container swap built successfully on every configuration.

@zvonand

zvonand commented Oct 25, 2026

Copy link
Copy Markdown
Member Author

@blau-ai

@blau-ai

blau-ai commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator

CI triage for #2638 (backport of upstream ClickHouse#86933, MarkRanges: devector over deque)

Verdict: 6 red checks — 0 caused by this code change. 1 is a DCO/sign-off formality, the rest are pre-existing/infra/flaky failures that also hit the antalya-25.8 base branch (every recent merge into it is red). The actual diff is clean: it only touches the MergeTree read path (swaps the MarkRanges base container std::deque → boost::container::devector and adjusts three iterators to auto * since devector iterators are raw pointers). All 11 builds, all 4 unit-test variants, the Fast test, and every stateless suite including all S3-storage ones passed.

Check Classification Cause
DCO ⚠️ process Missing Signed-off-by on the upstream merge commit
Stateless tests (amd_ubsan, parallel) ✅ not PR Pre-existing ReplicatedMergeTree assertion crash
Stress test (amd_msan) ✅ not PR Hybrid engine startup dependency-order bug
RegressionTestsAarch64 / S3 (azure, 1) ✅ not PR (infra/flaky) Could not retrieve failing test; no S3 report
SourceUpload ✅ not PR (infra) Known-flaky source-upload step on this branch
PR — Aggregate gate; just rolls up the above

Details

DCO — the head commit e93b59b ("Merge pull request ClickHouse#86933 from azat/…", author Azat Khuzhin) has no Signed-off-by: line, so cncf/dco2 fails. This is the only item the author needs to act on, and it's purely administrative — not a code problem. Add a sign-off (e.g. git commit --amend -s / re-push with sign-off, per Altinity's DCO policy).

Stateless tests (amd_ubsan, parallel) — reported as "test runner terminated unexpectedly". The server aborted on a logical error, and the test 02344_distinct_limit_distiributed was merely the victim ("Reason: server died / Connection refused"). Decisive line:

<Fatal> : Logical error: 'Part all_110_110_0 from table test_8ew2jlfi.alter_table0 (...) remains in ZooKeeper after DROP_RANGE all_0_110_999999999_999999999'.

The abort is in StorageReplicatedMergeTree::paranoidCheckForCoveredPartsInZooKeeper (StorageReplicatedMergeTree.cpp:2775, via executeDropRange) — a replicated-table ZooKeeper sanity check in a background queue thread. This code path has nothing to do with MarkRanges/read-path marks. Not PR-caused; safe to re-run.

Stress test (amd_msan) — "Cannot start clickhouse-server". The server fails to load metadata because the Altinity-specific Hybrid engine can't attach a table whose segment table doesn't exist yet at load time:

Load job 'load table test_13.test_hybrid_unqualified' failed: Code: 36 ...
table 'test_13.test_hybrid_unqualified_segment' does not exist:
Cannot attach ... ENGINE = Hybrid(remote('localhost:9000', 'system.numbers'), number = 0, test_13.test_hybrid_unqualified_segment, number >= 10)

This is a load-ordering/dependency bug in the Hybrid engine (Altinity-specific), unrelated to this diff. Not PR-caused.

RegressionTestsAarch64 / S3 (azure, 1) — the Actions annotation is only "Process completed with exit code 1" and the job log isn't downloadable via gh (stream CANCEL); no praktika/regression report was published at the expected S3 path (404), so I could not identify the failing test name and won't over-claim. That said: all S3-storage stateless suites passed, this Azure suite depends on external cloud connectivity, and the base branch is red on every merge — so this is most likely infra/flaky rather than from a read-path container swap. Suggest a re-run; if it persists, it needs a human to pull the regression logs.

SourceUpload — infra step (source tarball upload); log not downloadable. It's a known-flaky area on antalya-25.8 (recent base merges include #2635 25.8-antalya-sourcefix and #2642 submodule-cache targeting exactly this). Not PR-caused; re-run.

Bottom line

Nothing in the C++ change needs fixing — it builds everywhere and passes all functional/unit/S3 tests. The only author action is adding the DCO sign-off; the remaining red checks are pre-existing/infra and can be re-run (ideally after the base branch's own CI is green again).

@zvonand
zvonand merged commit c43a24c into antalya-25.8 Oct 25, 2026
518 of 527 checks passed
@zvonand zvonand added the 25.8 25.8 Altinity Stable label Oct 25, 2026
@zvonand zvonand added the verified Approved for release label Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

25.8 25.8 Altinity Stable antalya-25.8.33.20003 backport Backport verified Approved for release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants