Repository navigation
Metrics for current effective kafka topic config - #12
Merged
Merged
Conversation
…nfig
A Buckety's parameters do not say what retention a topic has. The broker merges topic overrides
with its own config and built-in defaults: a topic whose spec leaves retention.ms out keeps records
forever on a Kafka with log.retention.hours=-1 and for 7 days on a default Redpanda, and a config.*
key removed from the spec stays on the topic as an unmanaged override.
Two gauges, set from the DescribeConfigs answer every reconcile already reads (no extra broker
call, as fresh as the re-check cadence):
buckety_kadm_topic_config{namespace,buckety,topic,key,source}
buckety_kadm_topic_cleanup_policy{namespace,buckety,topic,policy,source} 1
for retention.ms, retention.bytes, delete.retention.ms, min/max.compaction.lag.ms,
min.cleanable.dirty.ratio, segment.ms, segment.bytes, max.message.bytes and min.insync.replicas.
source is where the broker says the value comes from: topic (an override), broker (broker or
cluster config), default (built in) or unknown. An alter the broker acknowledged is exported at
once as source=topic; a source or policy change replaces its series; a key the broker does not
report has none. Configuration only: the topic's state (sizes, offsets, partitions, replicas)
stays with Kafka exporters.
Series go when their Buckety does. registry.ForgetBuckety is a new optional driver capability the
reconciler calls on every backend once a Buckety is NotFound (its backend is no longer readable,
and DeleteBuckety only runs for retentionPolicy=Delete). kadm's implementation also drops the
change counters, which until now stayed until the next restart.
e2e parameter-mutation asserts the requested retention.ms as a topic override before and after the
patch, and an inherited retention.bytes; test/e2e/lib.sh gains controller_metric. Run locally
against Redpanda v26.1.9: an unconfigured topic exported all ten keys but min.insync.replicas, each
source=default (retention.ms 604800000), and no buckety_kadm_* series remained after the scenario
namespace was deleted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
scripts/bump-release.sh with the go.mod toolchain (go1.27.1). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Except partitions and replication factor that are already available with tools like kminion.