Repository navigation
Pass 2: shared window grids (group-wide panes and long-window segments) #731
Description
Problem, by example
Three PromQL panels over the same metric, all exact sums:
| Query | Repeats every |
|---|---|
sum_over_time(m[5m]) |
1 min |
sum_over_time(m[1h]) |
5 min |
sum_over_time(m[365d]) |
1 day |
Today: the window-composition rule in Pass 2 (pane_candidates / window_composition_pass2_candidates) offers each query its own tumbling-pane chain, with pane width = its own repeat interval. That gives 5 × 1-min panes, 12 × 5-min panes and 365 × 1-day panes. These are three separate builds over the same rows, and every arriving row updates all three.
Expected: Pass 2 also offers shared window grids, and Stage 3 picks the cheapest:
- Group-wide panes: one grid at the greatest common divisor of the group's pane widths (here 1 min), which the 5m and 1h windows both merge from.
- Long-window segments: for long windows, coarse equal-width segments (e.g. 1-day or 1-year panes) that the year-long window merges from, instead of 365 or more fine panes.
Each query's per-query panes stay as candidates too.
Cause
Only per-query panes exist. A grid shared across queries, and segments for long windows, are not implemented.
Proposal (approved 2026-10-14)
- Equal-width grids only. There is no end mask: an uneven segment such as 2y is stored as two 1y panes.
PaneChainstays{ pane_width, pane_count }. - The group-wide pane width is an extra Pass 2 candidate beside the per-query panes, and Stage 3 picks.
- One "shared window grid" rule covers per-query panes, group-wide panes and long-window segments, with one coverage proof.
- Grid candidates are limited to the deduplicated set {per-query, group gcd, segment grid}.
Earlier work is in draft PRs #628 (shared year-aligned segments), #632 (TimeShift/TimeRange pricing) and #634 (the planner chooses the segment grid). They predate the current stage code, so they should be reused for ideas rather than merged.
Scope / acceptance
- For the example above, Pass 2 lists the group-wide 1-min grid and a segment grid in addition to the three per-query chains. The coverage proof accepts them, and Stage 3 prices them.
- With one query per metric, Examples 1–4 select the same plans at the same costs as today.
Status / related
Deferred on 2026-10-16. Sliding windows and the Exponential Histogram are tracked separately in #522. Related draft PRs (left open): #628, #632, #634.
Activity
zzylol commentedon Oct 21, 2026
Example from the RangeHeavy evaluation workload (8 users): queries at 60 s and 300 s over one window
Two repeating PromQL queries over the same window, as in the 8-user RangeHeavy workload (users 0–2 and 7 repeat every 60 s, users 3–6 every 300 s):
{"query_workload": {"language": "promql", "query_batch": null, "repeating_queries": [ {"query": "quantile_over_time(0.5, data[15m])", "demand": {"fixed_interval": 60000}, "requirements": {"accuracy": {"max_error": 0.05, "failure_probability": 0.01, "accuracy_scope": "whole_result"}}, "predictability": {"predictable": {"known_at": null}}}, {"query": "quantile_over_time(0.5, data[15m])", "demand": {"fixed_interval": 300000}, "requirements": {"accuracy": {"max_error": 0.05, "failure_probability": 0.01, "accuracy_scope": "whole_result"}}, "predictability": {"predictable": {"known_at": null}}}]}, "data_workload": {"arrival": {"ingesting": {"rows_per_second": 2000000.0}}, "sources": {"data": {"series_count": 10000, "metric_type": "gauge", "values_non_negative": true, "sample_interval": 5}}}}Today: the window-composition rule gives each query its own chain with pane width = gcd(window, its own interval): 15 × 1 min panes for the 60 s query and 3 × 5 min panes for the 300 s query. Every arriving sample updates both chains.
Expected: Pass 2 also offers one chain at the gcd of the two pane widths (1 min) that both read: the 60 s query merges its 15 panes every minute, the 300 s query merges the same 15 panes every 5 minutes. Stage 3 picks.
Size of the gap. In the 8-user workload each 300 s component (quantiles and sums at 15 m, 1 h, 6 h and 24 h) keeps its own chains next to the 60 s component over the same window: 0.13 + 0.00 + 0.42 + 0.01 + 2.35 + 0.08 + 9.29 + 0.33 = 12.6 GB of average memory (the evaluation's per-component Stage 3 numbers), all of which the 60 s chains already cover.
Scope of the next slice: the group-wide pane width of this issue, for builds whose windows are equal or nested and end at the same time, read through the merge-only-the-newest-panes field (
SummaryMerge.newest_panes). The shared width is the gcd of the members' own pane widths, offered only when it is not finer than the finest member's own panes (the shared chain never has more panes per window than some query already builds). Long-window segments stay open.