Skip to content

Stable 25.8 Backport of #86933 - RFC: Optimize MarkRanges by using devector over deque - #2637

Merged
mkmkme merged 1 commit into
stable-25.8from
backports/stable-25.8/86933
Oct 24, 2026
Merged

mkmkme merged 1 commit into
stable-25.8from
backports/stable-25.8/86933

Conversation

@zvonand

@zvonand zvonand commented Oct 22, 2026

Copy link
Copy Markdown
Member

Backport of ClickHouse#86933

Changelog category (leave one):

  • Performance Improvement

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Improves performance of fast queries with lots of parts in table (by optimizing MarkRanges by using devector over deque) (ClickHouse#86933 by @azat)

RFC: Optimize MarkRanges by using devector over deque
@zvonand zvonand added the backport Backport label Oct 22, 2026
@github-actions

Copy link
Copy Markdown

Workflow [PR], commit [8851860]

@mkmkme

mkmkme commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator

@blau-ai

@blau-ai

blau-ai commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator

CI triage — 5 red checks, 0 caused by this PR

This is a backport of upstream #86933 (RFC: Optimize MarkRanges by using devector over deque). The diff only swaps the MarkRanges base container std::deque → boost::container::devector and adjusts a few iterators to pointers. Nothing else changed.

Verdict: 3 flaky stateless tests + 1 flaky Keeper-regression shard + DCO. None point at the MarkRanges change.

The decisive argument: the MarkRanges container is on the hot read path of every MergeTree query, so a real regression would light up read-path tests across the board. Instead every other suite is green — Fast test, all 8 Builds, all Unit tests, all Integration tests, Stress, AST fuzzer, BuzzHouse, and the stateless suites in debug / asan / msan / ubsan / release / arm. Only 3 isolated single-test failures remain, all in the slowest (TSan) variants, and none exercise mark-range reading.

Stateless tests (amd_tsan, parallel, 2/2) — flaky, not PR-related

02597_column_update_tricky_expression_and_replication

Code: 999. Coordination::Exception: Operation timeout, path /clickhouse/tables/.../mutations/0000000000. (KEEPER_EXCEPTION)
(query: KILL MUTATION ... SYNC)

A ZooKeeper/Keeper coordination timeout on KILL MUTATION ... SYNC — a classic replication-timing flake under TSan load, unrelated to MergeTree mark ranges. → safe to re-run.

Stateless tests (amd_tsan, s3 storage, parallel) — flaky, not PR-related

04043_system_asynchronous_inserts_user_filter

 secret_user sees:
-1
+0
 admin sees:
-1
+0

The async INSERT hadn't yet appeared in system.asynchronous_inserts when the SELECT ran — a race in the test's own timing (async-insert registration), nothing to do with reads. → safe to re-run.

Stateless tests (amd_tsan, s3 storage, sequential, 2/2) — flaky, not PR-related

03321_clickhouse_local_initialization_not_too_slow_even_under_sanitizers

-Ok
+Fail: 9

This is a startup-latency budget test; it managed only 9 clickhouse-local iterations instead of ≥10 within the time window on a loaded TSan+s3 runner. It is a timing/perf test by design and is a known flake under sanitizer load. → safe to re-run.

RegressionTestsRelease / ClickHouseKeeper (2, no_ssl) — flaky/infra, not PR-related

Only this one shard failed (exit code 1, no S3 report beyond that). The identical shard passes on aarch64 (ClickHouseKeeper (2, no_ssl)), and every other Keeper shard — release (1,no_ssl), (1,ssl), (2,ssl) and all four aarch64 — is green. A single asymmetric Keeper-suite failure, on a component that does not use MarkRanges at all, is infra/flake. → safe to re-run.

DCO — expected for a backport, needs a maintainer action

The backported upstream commits carry no Signed-off-by trailers (upstream ClickHouse doesn't use DCO), so cncf/dco2 flags them. This is not a code problem. Fix is one of the usual backport options — add the DCO override label, or sign off the commits (git rebase --signoff). I'm not touching this: it would mean rewriting others' commit history / force-pushing, which is outside what I do.


Suggested next step: re-run the 3 TSan stateless jobs and the single Keeper shard (they should go green), and apply your usual DCO handling for backports. No code change to the PR is warranted.

@mkmkme mkmkme added the verified Approved for release label Oct 24, 2026
@mkmkme
mkmkme merged commit 06c5722 into stable-25.8 Oct 24, 2026
748 of 756 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants