[subkanthi]

Changelog category (leave one):

• New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Add new Unity catalog implementation with Delta Lake and Iceberg support(ClickHouse#115879 by @scanhex12)

CI/CD Options

Exclude tests:

• Fast test
• Integration Tests
• Stateless tests
• Stateful tests
• Unit tests
• Performance tests
• Aarch64 tests
• All with ASAN
• All with TSAN
• All with MSAN
• All with UBSAN
• All with Coverage
• All Regression
• Disable CI Cache

Regression jobs to run:

• Fast suites (mostly <1h)
• Aggregate Functions (2h)
• Alter (1.5h)
• Benchmark (30m)
• CAS (content-addressed storage; Antalya only)
• ClickHouse Keeper (1h)
• Iceberg (2h)
• LDAP (1h)
• OAuth (5m)
• Parquet (1.5h)
• RBAC (1.5h)
• SSL Server (1h)
• S3 (2h)
• S3 Export (2h)
• Swarms (30m)
• Tiered Storage (2h)

[github-actions]

Workflow [PR], commit [cdbcf6d]

[subkanthi]

Databricks Setup Instructions
https://gist.github.com/subkanthi/cfdae61bb674f35d90da5d008802c871

TESTING

CREATE DATABASE unity_pat_token
ENGINE = DataLakeCatalog('https://2892208872697955.5.gcp.databricks.com/api/2.1/unity-catalog')
SETTINGS catalog_type = 'unity', use_unity_catalog_v2 = 1, warehouse = 'mlops', catalog_credential = 'access_token'

Query id: e087bd0b-b4bc-4b2e-b5a3-c2b77e202d4d

Ok.

0 rows in set. Elapsed: 0.070 sec. 

Ubuntu-2404-noble-amd64-base :) show tables;

SHOW TABLES

Query id: 3ab50d61-d73b-4707-8f36-ffa92f3a48d9

   ┌─name─────────────────┐
1. │ default.people_delta │
2. │ mlops_dev.housing    │
   └──────────────────────┘

2 rows in set. Elapsed: 2.762 sec. 

[zvonand]

@blau-ai

[blau-ai]

CI triage — PR #2607 (Unity catalog v2 backport → antalya-26.8)

Verdict: 79 red checks, and none of them are caused by this PR's code. The PR only touches src/Databases/DataLake/* and the test_database_unity_v2 / test_database_delta integration tests; every failure is in an unrelated subsystem and matches known pre-existing / infra / flaky patterns. (PR is already merged — this is a retrospective read, no action needed on the diff.)

Breakdown of the 79: 1 integration shard, 1 PR aggregate gate (red only because its children are red — no error of its own), and 76 regression-suite rows — which are really ~38 suites each reported twice: once as Regression release … (S3 report) and once as RegressionTestsRelease / … (Actions). Details per group:

1. Integration tests (amd_asan_ubsan, db disk, old analyzer, 3/8) — NOT PR-caused (flaky)

Only one test failed in the whole praktika PR report (1/766):

FAIL  test_s3_cluster/test.py::test_graceful_shutdown
      test_s3_cluster/test.py:2171 - assert errors == 0
E     assert 2 == 0

Timing assertion on error count during node shutdown, in test_s3_cluster — a subsystem this PR doesn't touch. Classic flaky. Safe to re-run. The PR's own new tests (test_database_unity_v2, test_database_delta) are not among the failures — they passed.

2. Regression suite — NOT PR-caused (pre-existing / environment / flaky)

Multiple independent, pre-existing issues, none in DataLake code:

• cas, all cas_*, s3_cache_*, tiered_storage_cas (the bulk of the rows) — the server fails to start:

  Code: 137. DB::Exception: MetadataStorageFactory: unknown metadata storage type: cas.
  (UNKNOWN_ELEMENT_IN_CONFIG)
  

  The regression environment configured a cas metadata-storage type the merge-commit binary doesn't register, so every CAS suite's server dies on startup and the whole suite goes red. This is a framework/binary-config mismatch for this run, not a code defect — and CAS metadata storage has nothing to do with data-lake catalogs. (Sibling merged PRs Antalya 26.8: Forward auth tokens to data lake catalogs #2641/Antalya 26.8: auto-grp-pr-2308: combined port of 2 PRs #2658 did not hit this, i.e. it's specific to this run's environment, not to the Unity code.)

• swarms (Regression release swarms / RegressionTestsRelease / Swarms) — fails on every recent antalya-26.8 PR (confirmed red on Antalya 26.8: Forward auth tokens to data lake catalogs #2641 and Antalya 26.8: auto-grp-pr-2308: combined port of 2 PRs #2658 too). Known pre-existing / flaky suite. Safe to ignore here.

• oauth — 8 fail / 15 OK, failing with status_code=500 and Not enough privileges … CREATE/SELECT grant errors in the token-auth mock. Unrelated to this PR; the OAuth/IdP area is in active flux (e.g. Antalya 26.8: Enable Google as IdP for token-based authentication #2632) and these are flaky. Re-run.

• settings — whole-suite collapse (~3712 fail / ~1 OK), generic AssertionError: Oops! Assertion failed, same "server/environment didn't come up" signature as the CAS group rather than a targeted settings-coverage miss (which would be a handful of fails). Environment, not the diff.

• clickhouse_keeper_no_ssl_1, s3_azure_1 — same class of unrelated infra/flaky regression noise; none exercise DataLake catalog code.

Suggested next steps

• Nothing to fix in the PR — no failure maps to the changed files. The diff is clean w.r.t. CI.
• The integration + oauth/swarms/keeper/s3_azure failures are re-run candidates (flaky/pre-existing).
• The CAS / settings whole-suite collapses are a regression-environment config/binary mismatch for this run, worth a note to whoever maintains the antalya regression pipeline — but not a signal about this code.

Evidence: praktika result_pr.json for cdbcf6d (only test_graceful_shutdown failed); regression report.htmls for cas/oauth/settings; gh pr checks on sibling PRs #2641/#2658 for the swarms baseline.