[zvonand]

Backport of ClickHouse#85535

Changelog category (leave one):

• Performance Improvement

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Up to 8x faster SELECT queries with heavy partition pruning on tables with 10K+ parts (ClickHouse#85535 by @jawm)

[github-actions]

Workflow [PR], commit [244e825]

[zvonand]

@blau-ai

[blau-ai]

CI triage for #2639

Verdict: no PR-caused code failures. 5 red checks → 0 caused by this PR, 1 flaky test, 2 infra, 1 base-image scan, 1 DCO sign-off (process). The actual C++ change (MergeTree part-list locking + ReadFromMergeTree part passing) is healthy per CI — see the health check at the bottom.

Check	Classification	Cause
Stateless tests (amd_msan, parallel, 2/2)	Flaky, not PR	timing race in one async-insert test
Regression aarch64 S3 (aws_s3, 2)	Infra, not PR	30s testflows timeout on real AWS S3
RegressionTestsRelease / SSLServer (1)	Infra, not PR	self-hosted runner lost network
GrypeScanServer / Grype Scan	Infra / pre-existing, not PR	base-image CVEs (0 high/critical)
DCO	Process, not code	upstream merge commit lacks Signed-off-by
PR	aggregate gate	red only because of the above

Details

1. Stateless amd_msan, parallel, 2/2 — flaky, unrelated. The only failing test is 04043_system_asynchronous_inserts_user_filter:

 secret_user sees:
-1
+0
 admin sees:
-1
+0

The test does INSERT ... --wait_for_async_insert 0 --async_insert_busy_timeout 600000, then immediately SELECT count() FROM system.asynchronous_inserts with no polling/retry. It races the entry landing in the in-memory async-insert queue; under MSan's heavy instrumentation the SELECT ran before the entry was registered, so it saw 0 instead of 1 (restricted_user correctly saw 0). This is the async-insert queue subsystem, which this PR does not touch. It passed on the other 10 stateless configs (asan/tsan/debug/ubsan/arm + msan 1/2). → Safe to re-run.

2. Regression aarch64 S3 (aws_s3, 2) — infra timeout. Error is testflows.uexpect.ExpectTimeoutError: Timeout 30.000s in orphans/full replication/detach and drop → delete_replica (a clickhouse-client prompt that didn't return in 30s) against real AWS S3. The minio and gcs regression variants on the same commit all passed — if the part-selection change had broken S3 reads, those would fail too. → Infra/flaky, re-run.

3. RegressionTestsRelease / SSLServer (1) — dead runner. Check annotation is explicit: "The self-hosted runner lost communication with the server." SSLServer shards 2–3 and all three aarch64 SSL shards passed. → Infra, re-run.

4. GrypeScanServer / Grype Scan — base-image scan. Reports 0 high/critical vulnerabilities; the red status comes from medium/low CVEs in the server image's OS packages. The -alpine server and Keeper Grype scans passed. A C++ source change cannot introduce these. → Pre-existing/infra, not blocking on code.

5. DCO — sign-off missing. The upstream merge commit 8c8444f ("Merge pull request ClickHouse#85535") has no Signed-off-by line (your own 244e825 is fine — skipped for members). This is a metadata/process check, not a code issue. Fix options: add a DCO remediation commit, or git rebase --signoff stable-25.8 && git push (history rewrite — only the commit author should do this), or have a maintainer override the check. I did not attempt it myself: I won't force-push/rewrite history, and I can't legitimately sign off on another author's commit.

Health check of the PR

The code change (lockParts() → readLockParts() in parts collection / DataPartsExchange, std::mutex → SharedMutex for the data-parts lock, and passing the part list as a shared_ptr into ReadFromMergeTree) is the kind of concurrency change where races would surface in sanitizer and stress runs — and all of those are green:

• All 7 builds green; Fast test green.
• Unit tests asan/tsan/msan/ubsan — all pass.
• All stateless suites green except the single unrelated flake above.
• All integration tests (asan old-analyzer, binary, tsan, arm distributed-plan) green.
• Stress tests (debug/tsan/ubsan/msan) — "No errors found"; TSan parallel + stress clean, which is the strongest available signal that the new shared/read-lock split introduces no data race.
• AST fuzzer and BuzzHouse (debug/tsan/msan/ubsan) — all OK.

Recommendation: re-run the three flaky/infra jobs (msan 2/2 stateless, aarch64 aws_s3, Release SSLServer 1); resolve DCO by signing off the merge commit or via maintainer override. No code change is needed for the failures — none of them implicate this PR.

_{🤖 Triage by @blau-ai from CI evidence (praktika result_pr.json, regression reports, check-run annotations). I did not build or run tests locally.}