[xieandrew]

Closes #2182.

Changelog category (leave one):

• Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Improve iceberg write compatibility for Unity and other REST catalogs. Fixes writing a corrupted manifest list when the existing manifest list written externally has a key_metadata field.

Documentation entry for user-facing changes

...

CI/CD Options

Exclude tests:

• Fast test
• Integration Tests
• Stateless tests
• Stateful tests
• Unit tests
• Performance tests
• Aarch64 tests
• All with ASAN
• All with TSAN
• All with MSAN
• All with UBSAN
• All with Coverage
• All Regression
• Disable CI Cache

Regression jobs to run:

• Fast suites (mostly <1h)
• Aggregate Functions (2h)
• Alter (1.5h)
• Benchmark (30m)
• CAS (content-addressed storage; Antalya only)
• ClickHouse Keeper (1h)
• Iceberg (2h)
• LDAP (1h)
• OAuth (5m)
• Parquet (1.5h)
• RBAC (1.5h)
• SSL Server (1h)
• S3 (2h)
• S3 Export (2h)
• Swarms (30m)
• Tiered Storage (2h)

[github-actions]

Workflow [PR], commit [4af4c5a]

[xieandrew]

@blau-ai

[blau-ai]

CI triage for 11d1a51

Verdict: 1 failing check, 0 caused by this PR. Stateless tests (amd_debug, parallel) fails on 5 test timeouts. They are on dataset and network-heavy tests that have nothing to do with Iceberg, and the same tests are timing out right now on unrelated antalya-26.3 backports. All other checks marked "skipping" in the GitHub UI were dropped because of this one failure (Dropped due to previous failure [Stateless tests (amd_debug, parallel)]) or excluded by the arm/tsan/msan tag filter. They didn't run and didn't fail.

Check	Result	Classification
Stateless tests (amd_debug, parallel)	5 tests timed out (each one failed in both the main run and the retry, giving the "Failed: 10" count)	Infra / pre-existing on base, not PR-related
Build (amd_debug/asan/ubsan/binary/release, arm_release), Fast test, Quick functional, Stateless amd_asan 1–4/4, Unit tests asan/ubsan, Docker images	✅ pass	—
Integration, Stress, AST fuzzer, BuzzHouse, other stateless suites	⏭ dropped after the failure above	Not run, so they say nothing about this PR yet

Stateless tests (amd_debug, parallel)

Failing tests (report):
04033_tpc_ds_q09, 04033_tpc_ds_q14, 03634_autopr_output_bytes_estimation, 00152_insert_different_granularity, 00154_avro. All hit the 600 s test timeout or the client's 300 s receive timeout. In the first pass, 04033_tpc_ds_q24/q34/q67/q23 and 03634_autopr_input_bytes_estimation also timed out but passed on retry.

Why it isn't the PR: the hung-query check at the end of job.log shows queries such as SELECT * FROM test.hits WHERE URL LIKE '%google%' ... stuck inside DNS resolution during a TLS connect:

clickhouse::__poll
clickhouse::__GI___res_context_send
clickhouse::_nss_dns_gethostbyname4_r
clickhouse::getaddrinfo
Poco::Net::DNS::hostByName(...)                      base/poco/Net/src/DNS.cpp:80
Poco::Net::SecureSocketImpl::isLocalHost(...)
Poco::Net::SecureSocketImpl::verifyPeerCertificateImpl(...)
...
DB::EndpointConnectionPool<Poco::Net::HTTPSClientSession>::prepareNewConnection(...)

• The tests that timed out all read remote datasets over HTTPS (test.hits, TPC-DS tables, avro and granularity fixtures). The runner's DNS stalled and these queries waited until the timeout. That is an infra/network symptom.
• The PR only touches Iceberg write paths, RestCatalog.cpp, and a single blank line in IKeeper.h. None of these tests exercise those paths.
• The same failure shows up on unrelated sibling PRs on the same base today:
  • Antalya 26.3 Backport of #102156 - Fix Nullable(X)->Y cross-type ALTER MODIFY COLUMN #2712 (Nullable ALTER fix): Stateless tests (amd_debug, parallel) failed with 13 failures: 04033_tpc_ds_q09/q14/q24, 00154_avro, 03634_autopr_output_bytes_estimation, 00152_insert_different_granularity, … (a superset of ours).
  • Antalya 26.3 Backport of #127318 - Distrust the compromised eMudhra certificates #2697 (eMudhra certificates): failed with 04033_tpc_ds_q09/q14, 03634_autopr_input_bytes_estimation, 00071_merge_tree_optimize_aio.
• The 2 BROKEN tests (03441_deltalake_*) are expected-broken and don't count toward the failure.

Next step: re-run the failed jobs. No code change is needed on this PR. If it times out again, the problem is the runner's DNS/egress on the antalya-26.3 stateless runners and needs someone with CI access, not a change to this backport. I found no open tracking issue for it, so filing one may be worth it, given that at least three PRs are affected.

Caveat: because of this failure, the integration tests (where the Iceberg/REST-catalog coverage lives), the stress tests and the fuzzers have not run on this PR. Don't take the green checks as proof the backport is correct until a re-run gets past this stage.

_{Evidence: praktika result_pr.json and job.log for 11d1a51, plus result_pr.json for #2712 (ede57c1) and #2697 (3c607d2).}

[subkanthi]

Medium — write.data.path only on IcebergStorageSink
Yes, from this PR’s change. The PR adds setDataLocation / write.data.path and only wires it in IcebergStorageSink. IcebergImportSink was left untouched, so the new property is applied inconsistently. Same gap exists in upstream #98162.

Low — stale TODO above fail-after-retries
Caused by this PR’s edit. The fail-closed throw is new; the old “perhaps we should fail” TODO was not removed/updated, so the comment became wrong.

@xieandrew if these are valid maybe we can create separate issues to track these.

[xieandrew]

@subkanthi I created an issue for write.data.path specifically for EXPORT PART/PARTITION here #2757. This issue still exists in 26.6 and 26.8. There are other instances of this but they affect non-Antalya upstream as mentioned.

For the second issue the comment is still on other branches like 26.6 and 26.8. Removing it in this PR would deviate from other branches for no functional difference.