Skip to content

Stable 23.8 - Bump brotli to v1.0.10, update runner labels, use minio mirror image - #1864

Open
strtgbb wants to merge 14 commits into
releases/23.8.16from
update-runner-labels-23.8
Open

Stable 23.8 - Bump brotli to v1.0.10, update runner labels, use minio mirror image#1864
strtgbb wants to merge 14 commits into
releases/23.8.16from
update-runner-labels-23.8

Conversation

@strtgbb

@strtgbb strtgbb commented Jun 2, 2026 •
edited by zvonand
Loading

Copy link
Copy Markdown
Collaborator

Changelog category (leave one):

  • Critical Bug Fix (crash, data loss, RBAC)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Bump brotli from v1.0.9 to v1.0.10; fix CVE-2026-111497

Description

CI fixes for releases/23.8.16:

  • Update runner labels; read CI secrets from env instead of SSM; update artifact actions.
  • Use the clickhouse/minio-minio mirror image for integration tests. minio/minio was deleted from Docker Hub, so every with_minio=True test failed in cluster.start() with pull access denied for minio/minio. The mirror (upstream Mirror the MinIO integration-test images into the clickhouse/ namespace ClickHouse/ClickHouse#119999) has no RELEASE.2021-09-23T04-46-24Z tag, so minio is bumped to RELEASE.2024-07-31T05-46-26Z, same as releases/24.8.14.
  • Take the test_storage_delta fix from upstream Upgrade MinIO to support accepting non signed requests ClickHouse/ClickHouse#55245 (the 2021 → 2023 minio upgrade): newer minio does not match a list_objects prefix with a leading slash.
  • Take the stateless-test minio/mc binaries from clickhouse/minio-minio:RELEASE.2024-07-31T05-46-26Z instead of dl.min.io, which now returns 410 Gone, so the image could no longer be rebuilt. Same as upstream Update minio in stateless tests ClickHouse/ClickHouse#67975 (stateless tests on 2024 minio), it also takes the renamed mc admin policy attach / mc anonymous set commands in setup_minio.sh and the S3ObjectStorage::listObjects fix that skips a "/" prefix.

CI/CD Options

Exclude tests:

  • Fast test
  • Integration Tests
  • Stateless tests
  • Stateful tests
  • Performance tests
  • All with ASAN
  • All with TSAN
  • All with MSAN
  • All with UBSAN
  • All with Coverage
  • All with Aarch64
  • All Regression
  • Disable CI Cache

Regression jobs to run:

  • Fast suites (mostly <1h)
  • Aggregate Functions (2h)
  • Alter (1.5h)
  • Benchmark (30m)
  • ClickHouse Keeper (1h)
  • Iceberg (2h)
  • LDAP (1h)
  • Parquet (1.5h)
  • RBAC (1.5h)
  • SSL Server (1h)
  • S3 (2h)
  • S3 Export (2h)
  • Swarms (30m)
  • Tiered Storage (2h)

@strtgbb strtgbb added stable cicd Improvements and fixes to the CICD process 23.8 labels Jun 2, 2026
@zvonand
CI: use the clickhouse/minio-minio mirror image
0a5d74b 
minio/minio was deleted from Docker Hub, so every with_minio=True test
fails in cluster.start() on the image pull. Point docker_compose_minio.yml
at the clickhouse/minio-minio mirror (upstream ClickHouse#119999). The mirror has no
RELEASE.2021-09-23T04-46-24Z tag, so bump to RELEASE.2024-07-31T05-46-26Z,
the same image releases/24.8.14 uses.

Also take the test_storage_delta change from upstream ClickHouse#55245 (the
2021 -> 2023 minio upgrade): newer minio does not match a list_objects
prefix with a leading slash.
@zvonand zvonand changed the title Stable 23.8 - update runner labels Stable 23.8 - update runner labels, use minio mirror image Oct 26, 2026
@zvonand
CI: take stateless-test minio binaries from the mirror image
e2bc825 
dl.min.io returns 410 Gone for every minio/mc binary, so the
stateless-test image can no longer be rebuilt. Copy minio and mc from
clickhouse/minio-minio:RELEASE.2024-07-31T05-46-26Z instead, the same
image the integration tests use.

The newer mc and minio need the same changes upstream made when it
moved stateless tests to 2024 minio (ClickHouse#67975): the renamed mc policy
commands, and not sending a "/" prefix in S3ObjectStorage::listObjects.
@zvonand
Bump brotli from v1.0.9 to v1.0.10
24742c7
@zvonand zvonand changed the title Stable 23.8 - update runner labels, use minio mirror image Stable 23.8 - Bump brotli to v1.0.10, update runner labels, use minio mirror image Oct 26, 2026
@zvonand zvonand mentioned this pull request Oct 26, 2026
Closed
5 of 30 tasks
@altinity-robot

altinity-robot commented Oct 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

This is an automated comment for commit b0eee6b with description of existing statuses. It's updated for the latest CI running

❌ Click here to open a full report in a separate page

Successful checks
Check nameDescriptionStatus
ClickHouse build checkBuilds ClickHouse in various configurations for use in further steps. You have to fix the builds that fail. Build logs often has enough information to fix the error, but you might have to reproduce the failure locally. The cmake options can be found in the build log, grepping for cmake. Use these options and follow the general build process✅ success
Compatibility checkChecks that clickhouse binary runs on distributions with old libc versions. If it fails, ask a maintainer for help✅ success
Docker image for serversThe check to build and optionally push the mentioned image to docker hub✅ success
Install packagesChecks that the built packages are installable in a clear environment✅ success
Push to DockerhubThe check for building and pushing the CI related docker images to docker hub✅ success
Ready for releaseThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Check nameDescriptionStatus
CI runningA meta-check that indicates the running CI. Normally, it's in success or pending state. The failed status indicates some problems with the PR⏳ pending
Integration testsThe integration tests report. In parenthesis the package type is given, and in square brackets are the optional part/total tests❌ failure
Mergeable CheckChecks if all other necessary checks are successful❌ failure
Stateful testsRuns stateful functional tests for ClickHouse binaries built in various configurations -- release, debug, with sanitizers, etc❌ error
Stateless testsRuns stateless functional tests for ClickHouse binaries built in various configurations -- release, debug, with sanitizers, etc❌ failure
Stress testRuns stateless functional tests concurrently from several clients to detect concurrency-related errors❌ failure

@zvonand
CI: fix test-util image build (LLVM key hash, cmake-data pin)
75fa109
@zvonand
CI: build server-jepsen-test on top of the PR's test-base image
62505bd 
server-jepsen-test was not listed as a test-base dependent, so it was built
from the stale test-base:latest with an outdated Kitware apt key.

Also push per-arch images without provenance attestation: an index with an
attestation is rejected by `docker manifest create`.
@zvonand
CI: fix docker_test unit tests
db4e8c2 
Restore the test images fixture and the expected changed images from
upstream, with altinityinfra names: the fixture referenced images it did
not define. Run test_get_changed_docker_images as a push to an existing PR,
since PR number 0 rebuilds every image. Mock reading the build log on
failure, and expect --provenance=false and the :latest tag in the build
command.
@zvonand
CI: run azurite on a newer Node.js in the stateless-test image
64375c6 
Azurite's transitive dependency @dabh/diagnostics >= 2.0.5 pulls
@so-ric/colorspace, which uses syntax Node.js 12 from Ubuntu 22.04 can't
parse, so azurite failed to start.
@zvonand
CI: log in to Docker Hub as altinityinfra in docker setup
2898a59 
DOCKER_PASSWORD belongs to the altinityinfra account (the image push
scripts log in with it), so pairing it with secrets.DOCKER_USERNAME failed
with 'incorrect username or password'.
@zvonand
CI: pin rustup to 1.27.0 in the binary-builder image
b9bce0e 
rustup >= 1.28 prints the toolchain list as '(active, default)', which the
bundled contrib/corrosion does not parse, so cmake failed with
"Could not find toolchain ''". Same pin as upstream 0511e72, plus
disabling auto self-update, since 'rustup toolchain install' otherwise
updates rustup back to the latest version.
@zvonand
Build rust crates with the workspace Cargo.lock
a2c307a 
Each crate is copied to the build directory on its own, outside of the
workspace, so cargo ignored rust/Cargo.lock and resolved the latest
dependency versions. Those now include crates using the 2024 edition
(cpufeatures 0.3.1), which the pinned nightly-2023-07-04 cargo can't parse.
@zvonand
CI: use the bitnamilegacy openldap image in integration tests
b0eee6b

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

23.8 23.8.16 cicd Improvements and fixes to the CICD process stable

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@strtgbb @altinity-robot @zvonand