Repository navigation
Expose metrics for topics configuration changes - #10
Merged
Merged
Conversation
Drivers may change an existing backend resource whenever the spec licenses it (kadm adds partitions and alters topic configs on every reconcile and recheck). That is intended, but until now it left no trace: an adopted topic whose spec disagrees with the broker gets modified silently. EnsureRequest gains an optional OnChange callback that drivers call once per acknowledged in-place change, plus the Buckety's identity so drivers can label their own metrics. The reconciler turns each change into a Normal Event (old -> new) and a structured log line. Events are not transition-gated: a change that repeats every recheck should show up repeatedly. Drivers that never call OnChange are unaffected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
alignTopic adds partitions when the Buckety asks for more than the
topic has, and sets every config.* key whose broker value differs, on
every reconcile and recheck. That behaviour stays. What changes is that
each such modification is now observable, so operators can alert when
the controller touches an adopted topic:
- buckety_kadm_partitions_added_total{namespace,buckety,topic}, by the
number of partitions added
- buckety_kadm_config_changes_total{namespace,buckety,topic,key}, once
per key set
- a registry.Change per modification (PartitionsAdded,
TopicConfigChanged) for the reconciler's Events and logs
All of these move only after the broker acknowledged the change. kadm
reports per-topic CreatePartitions rejections in the response rather
than the error, and alignTopic has always carried on past them; it
still does (no reconcile behaviour change), but a rejection is logged
instead of counted. Labels are identities only, never values.
The broker mutations go through a two-method interface satisfied by
*kadm.Client so unit tests can drive alignTopic without a broker.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A partition shrink request (or any driver's ErrParameterDrift) already
sets ParameterDrift=True and Ready=False, but only kubectl shows it.
Operators adopting existing topics need to alert on it.
buckety_parameter_drift{namespace,buckety,backend} is set at the end
of every reconcile from the condition the pass leaves behind, so it is
driver-independent and agrees with status whichever branch returned:
1 while True, 0 otherwise. The series is dropped when the finalizer is
released and on a NotFound reconcile, so deleted Bucketies do not keep
stale series. spec.backend is immutable, so a Buckety's label set is
fixed. Reconcile decisions are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
increase() and rate() need an earlier sample to see an increment. A counter series created by its own first increment has none, so an alert on increase(...[1h]) > 0 would never fire for the first partition increase or config alter a topic gets after a controller start. Touching the series for every aligned topic and managed config key makes later changes visible. Cardinality is unchanged: these are the same series the first change would create. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Describes the drift gauge and the two kadm change counters under Troubleshooting, next to the conditions and Events they complement, with a PrometheusRule to start from. The counter alerts are not a bare increase(...[1h]) > 0: the first reconcile of an adopted Buckety can create a counter series and increment it in the same pass, and increase() needs an earlier sample to see that, so the first change to a newly adopted topic would never fire. The "> 0 unless ... offset 1h" term covers series that are new within the window. Checked with promqltest: an adopted topic's first change alerts, a later change on a zero series alerts, an unchanged nonzero series does not. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
scripts/bump-release.sh at the head of the metrics changes, with the go.mod toolchain (go1.27.1). The same host reproduced main's pin, 20261002T051036Z@sha256:8c3c4937, byte for byte before this bump. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.