Skip to content

write __meta unconditionally to handle schema changes in existing cache - #207

Open
mlemnet wants to merge 3 commits into
PromPHP:mainfrom
mlemnet:fix/update-metadata-on-every-write
Open

mlemnet wants to merge 3 commits into
PromPHP:mainfrom
mlemnet:fix/update-metadata-on-every-write

Conversation

@mlemnet

@mlemnet mlemnet commented Sep 17, 2026

Copy link
Copy Markdown

Problem encountered:

When a metric's label schema changes (e.g., adding a new label to an existing counter), the Redis storage adapters s
ilently keep the old __meta, causing a fatal ValueError on the next render.

Root cause: updateCounter() only writes __meta when sAdd returns 1 (first registration). On subsequent w
rites, even with a different label set , sAdd returns 0 and __meta is skipped:

- Before (counter):
local added = redis.call('sAdd', KEYS[2], KEYS[1])
if added == 1 then
    redis.call('hMSet', KEYS[1], '__meta', ARGV[4])   - skipped forever after first write
end

updateGauge() has an equivalent guard that checks hSet/hIncrByFloat return values.

updateHistogram() does not have this problem, it writes __meta unconditionally on every call.

When old-schema samples coexist with new-schema samples in the same hash, RenderTextFormat::escapeAllLabels() call
s array_combine() with mismatched array lengths → ValueError that kills the entire scrape, not just the affe
cted metric.

This is a production issue for any application that changes metric labels across deployments (rolling updates, canar
y releases, feature flags). Fixes #147, fixes #179.

Reproduction:

  1. Register and increment a counter with labels ['foo']
  2. Without clearing storage, register the same counter with labels ['foo', 'bar'] and increment
  3. Call collect()__meta still says ['foo'] while the hash contains a ['val1', 'val2'] sample
  4. RenderTextFormat::render()ValueError: array_combine()

Proposed solution:

Remove the conditional gates on __meta writes in updateCounter() and updateGauge() for both AbstractRedis an
d RedisNg, making them unconditional, matching updateHistogram()'s existing behavior.

- After (counter):
redis.call('sAdd', KEYS[2], KEYS[1])
redis.call('hMSet', KEYS[1], '__meta', ARGV[4])   - always written

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant