Skip to content

feat(planner): tumbling-pane window composition as a Pass 2 rule (Example 3) - #713

Merged
zzylol merged 5 commits into
mainfrom
feat/g6b-pane-rule
Oct 15, 2026
Merged

zzylol merged 5 commits into
mainfrom
feat/g6b-pane-rule

Conversation

@zzylol

@zzylol zzylol commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Supersedes #601 (its window-form-as-Pass-1-axis design is replaced; #591/#592 landed in #696). Part of #509 (Example 3, Pattern B) and #580.

Why

#509's window-composition rule lets a repeating query build its summary as tumbling panes: quantile_over_time(0.99, latency_ms[5m]) every minute merges five 1-min KLLs. #696 added the family-capability table and made the executor run hand-written pane plans, but the planner never generated them. Agreed design for this step:

  • Window form is a Pass 2 rule in asap_aware_cse, not a Pass 1 axis.
  • Pane width is the per-query gcd(window, interval).
  • A pane chain is one compact node with no pane-count cap.
  • TimeShift is free and TimeRange is priced by the rows it keeps, in the generic per-operator pricing.

What

  • IR: ASAPOp::PaneChain. One node: a SummaryAgg's family, update, reduction, grouping and filter, built over pane_count back-to-back range windows of pane_width_ms, over the input a range selector would read. A SummaryMerge over the chain is the state of one build over the whole window:
    • coverage, schemas and timing treat it that way;
    • ir::pane_chain gives whole_window_build, pane(i) and expand_pane_chains.
  • Pass 2 window-composition rule (asap_aware_cse::{pane_sites, window_composition_variants}):
    • A pane site is a SummaryAgg over a range TimeRange(W) whose family has a merge in the family-capability table. Exact accumulators count; heap top-k counts, though it merges only approximately. The site must be read only by queries repeating at a fixed interval E.
    • Panes have width w = gcd(W, E); a site needs at least 2 panes.
    • Every non-empty subset of the sites is a candidate. Each chosen build becomes SummaryMerge(PaneChain).
    • The candidates go through the existing Stage 1 enumeration, so the 10^4 devtools limit counts them too. That enumeration is lazy across Pass 1 candidates only. Within one Pass 1 candidate, all of its Pass 2 variants are built eagerly, as the other rules' are: the 2^n − 1 pane subsets (n ≤ 20, otherwise an error candidate), deduplicated by a linear scan.
    • Order: identical-expression candidates, window-composition candidates, then each summary-capability candidate with its own; every window candidate is followed by its identical-expression candidates.
  • Guarantees. A read of merged panes gets the guarantee of the same read of one whole-window build when the family's merged error is Exact or SameAsUnionBuild. It gets none when the merged error is Approximate (evidence only) or the family has no merge.
  • Stage 3 pricing.
    • New PhysicalOperator::{TimeShift, TimeRange}: a shift costs nothing, and a range is priced by the rows it keeps.
    • A query-time scan reads the longest window above it, through any time shift.
    • A chain is priced as one build over its whole window plus one operation per row to place it in its pane, and keeps one state per group and pane. A merge costs one operation per state it reads.
    • The feat(planner): error-budget allocation and whole-result δ scaling (#669, part b) #712 sizing (with_resized_summaries, approximate_inputs, at_most_one_row) and the Count-Min weight check also cover chains.
  • Executor.
    • A planner-produced chain runs once expanded with expand_pane_chains, using feat(types): family-capability table and executor tumbling pane merges #696's merge path. compile rejects an unexpanded chain and says to expand it.
    • Expansion drops the chain's own materialization: the panes are new nodes, so expansion runs before timings are applied. Keeping panes between evaluations is G8's job.
    • Counter windows (review fix). A Rate or Increase evaluation's window is now the union of the range windows it reads, each moved back by the time shifts below it. Before, over expanded panes, the window was one pane: rate(m[5m]) gave 0.333 instead of 0.0667, and Increase extrapolated to the newest pane's start. The window also ends before a PromQL offset (ExactEvaluation::offset_ms). That fixes a pre-existing bug: rate(m[5m] offset 1m) gave 0.065 instead of 0.0667 even for one build.
  • stage1_logical_asap_optimization now takes the queries' recurrences. Pass2Candidate/Pass2Of gain panes (e.g. Kll for Q1 in 5 × 1m panes), which also appears in display names.

How

Pass 2 rewrites each chosen build SummaryAgg(TimeRange(W, X)) into SummaryMerge([PaneChain{child: X, …, w, W/w}]), keeping the build's output names, then composes every guarantee again. Coverage, schema and guarantee derivations all reduce a merged chain to whole_window_build, so no per-family logic is duplicated. Pane i is SummaryAgg(TimeRange(w, TimeShift(i·w, X))), with the query's own offset folded into the shift. Expansion turns each merge over a chain into a merge over these panes, newest first.

Key code interfaces added / modified

crates/types/src/ir/asap.rs, node.rs

/// An ASAP (summary) operator; `C` is how it refers to its inputs. The type stays;
/// `PaneChain` is new.
pub enum ASAPOp<C = Rc<OperatorNode>> {
    /// Builds one summary state per group over `child`'s rows.
    SummaryAgg { child: C, family: FieldDataType, input: SummaryUpdate, reduction: Reduction,
        grouping: GroupingStrategy, filter: Option<Predicate<C>> },   // unchanged
    /// Reads a query result (`query`) out of built summary state.
    SummaryEstimate { summary_input: C, query: SketchStatistic },   // unchanged
    /// Reads an exact accumulator's state as its finalized value.
    FinalizeExactAccumulator { child: C },   // unchanged
    /// Maintains the full declared population, including membership changes.
    MaintainPopulation { child: C, population: MaintainedPopulation<OperatorNode> },   // unchanged
    /// Reads an aggregate or top-k prefix from the maintained population.
    EvaluatePopulation { child: C, evaluation: PopulationStatistic },   // unchanged
    /// Merges compatible partial states of one grouping and family, including each
    /// pane of a `PaneChain`, into one state per group.
    SummaryMerge { children: Vec<C> },   // unchanged (doc now names PaneChain)
    /// Reserved, not implemented: subtracts one summary state from another.
    SummarySubtract { left: C, right: C },   // unchanged
    /// Reserved, not implemented: deletes `key` from a summary state.
    SummaryDelete { summary_input: C, key: ColumnId },   // unchanged
    /// Reserved, not implemented: joins two summary states on `key`.
    SummaryJoin { outer: C, inner: C, key: ColumnId, family: FieldDataType },   // unchanged
    /// Reserved, not implemented: a named deployment extension.
    Extension { child: C, name: String },   // unchanged
    /// One summary build split into `pane_count` back-to-back panes of `pane_width_ms`,
    /// newest first; a `SummaryMerge` over it equals one build over the whole window.
    PaneChain {                      // new: exhaustive matches on ASAPOp must add it
        /// The input a range selector would read; pane `i` covers `(-(i+1)·w, -i·w]` relative to it.
        child: C,
        /// The summary family each pane builds.
        family: FieldDataType,
        /// What each row contributes to the summary.
        input: SummaryUpdate,
        /// The aggregation the summary realizes.
        reduction: Reduction,
        /// How groups map to summary instances.
        grouping: GroupingStrategy,
        /// Rows kept before the build; `None` (default) keeps all.
        #[serde(default)]
        filter: Option<super::scalar::Predicate<C>>,
        /// Width of one pane, in ms.
        pane_width_ms: u64,
        /// Number of panes.
        pane_count: u32,
    },
}

impl OperatorNode {
    /// The SummaryAgg or PaneChain itself, or the first input's build under SummaryMerges.
    /// Used to read the definition behind a summary state; `None` for any other node.
    pub fn summary_build(&self) -> Option<&ASAPOp>;   // new
}

crates/types/src/ir/pane_chain.rs (new module)

/// The `PaneChain` a summary-state node carries merged (the chain itself, or the
/// one a `SummaryMerge` reads); `None` for any other node.
pub fn merged_chain(state: &OperatorNode) -> Option<&OperatorNode>;
/// The SummaryAgg over TimeRange(pane_count · pane_width_ms) the merged panes equal.
pub fn whole_window_build(chain: &OperatorNode) -> Result<OperatorNode, SchemaDerivationError>;
/// Pane `index` (0 = newest) as a plain SummaryAgg.
pub fn pane(chain: &OperatorNode, index: u32) -> Result<OperatorNode, SchemaDerivationError>;
/// Errors if a chain is read by anything but a `SummaryMerge`.
/// Every PaneChain under a SummaryMerge replaced by its panes; runs before the executor compiles.
pub fn expand_pane_chains(root: &Rc<OperatorNode>) -> Result<Rc<OperatorNode>, SchemaDerivationError>;

crates/asap-aware-mapping/src/asap_aware_cse.rs

/// One summary build the window-composition rule can build as tumbling panes: a
/// mergeable build over `TimeRange(W)` read only by queries repeating at fixed intervals.
pub struct PaneSite {                // new
    /// The `SummaryAgg` node to split.
    pub build: Rc<OperatorNode>,
    pub readers: Vec<usize>,         // query indices that read the build
    /// Pane width `w`, in ms.
    pub pane_width_ms: u64,          // gcd(window, intervals)
    /// `W / w` panes, at least two.
    pub pane_count: u32,
}

/// One Pass 2 candidate of the rule: each site in `sites` built as its panes and
/// merged before every read, guarantees composed again.
pub struct WindowCompositionVariant {   // new
    pub sites: Vec<usize>,              // indices into pane_sites
    /// The rewritten roots, or why composition failed (affects only this variant).
    pub roots: Result<Vec<QueryRoot>, PerComputationCandidateError>,
}

/// The most pane sites one candidate may have: every non-empty subset is a candidate.
pub const MAX_PANE_SITES: usize = 20;   // new

/// The pane sites of a candidate, children first; `interval_ms[q]` is query `q`'s
/// repeat interval, `None` when it does not repeat at a fixed interval.
pub fn pane_sites(roots: &[QueryRoot], interval_ms: &[Option<u64>]) -> Vec<PaneSite>;   // new
/// One variant per non-empty subset of `sites`; errors only for more than
/// `MAX_PANE_SITES` sites (too many subsets to enumerate).
pub fn window_composition_variants(                                                    // new
    roots: &[QueryRoot],
    sites: &[PaneSite],
) -> Result<Vec<WindowCompositionVariant>, PerComputationCandidateError>;

crates/asap-aware-mapping/src/pass/stage1_to_stage3.rs

/// Lazily enumerates every Stage 1 candidate: each Pass 1 candidate then its Pass 2 ones.
/// `recurrence[q]` says how often query `q` is evaluated, for the window-composition rule.
pub fn stage1_logical_asap_optimization<'a, Id: Clone>(
    per_computation: &'a PerComputationCandidates<Id>,
    recurrence: &[QueryRecurrence],              // new parameter
) -> impl Iterator<Item = Stage1Candidate> + 'a;

/// A Stage 1 candidate Pass 2 added, derived from one Pass 1 candidate.
/// The type stays; `panes` is new.
pub struct Pass2Candidate {
    /// The candidate's display id.
    pub id: String,                  // unchanged
    /// The id of the Pass 1 candidate it derives from.
    pub from_logical: String,        // unchanged
    /// Summary build sets made one build each (summary-capability rule).
    pub summaries: Vec<String>,      // unchanged
    /// Builds built as tumbling panes (window-composition rule).
    pub panes: Vec<String>,          // new: builds built as tumbling panes
    /// Identical node groups made one node (identical-expression rule).
    pub shared: Vec<String>,         // unchanged
}

/// What a Pass 2 candidate shares of a Pass 1 candidate. The type stays; `panes` is new.
pub struct Pass2Of {
    /// The Pass 1 candidate's position in the enumeration.
    pub from: usize,                 // unchanged
    /// The summary builds made one build each.
    pub summaries: Vec<String>,      // unchanged
    /// The builds built as tumbling panes.
    pub panes: Vec<String>,          // new
    /// The identical groups shared.
    pub shared: Vec<String>,         // unchanged
}

crates/asap-aware-mapping/src/analytical_cost.rs

/// Physical operator classes the analytical cost model prices. The type stays;
/// `TimeShift` and `TimeRange` are new.
pub enum PhysicalOperator {
    /// Reads the source rows.
    Scan,   // unchanged
    /// Keeps rows matching a predicate.
    Filter { predicate_operations_per_row: u64 },   // unchanged
    /// Computes expressions per row.
    Project { expression_operations_per_row: u64 },   // unchanged
    /// Groups rows by key and folds accumulators.
    HashAggregate { grouping_key_count: u64, accumulator_count: u64 },   // unchanged
    /// Sorts rows in memory.
    InMemoryComparisonSort { ordering_key_count: u64, partitioned: bool },   // unchanged
    /// Heap-based bounded ordering: keeps `limit + offset` rows.
    TopK { limit: u64, offset: u64, ordering_key_count: u64 },   // unchanged
    /// Equality hash join.
    HashJoin { build_side: HashJoinBuildSide, equality_key_count: u64 },   // unchanged
    /// Drops duplicate rows by key.
    HashDeduplicate { key_count: u64 },   // unchanged
    /// Concatenates inputs.
    Concat,   // unchanged
    /// SQL analytic window over ordered in-memory partitions.
    InMemoryAnalyticWindow { partition_key_count: u64, ordering_key_count: u64,
        function_operations_per_row: u64 },   // unchanged
    /// Returns at most `limit` rows after skipping `offset`.
    Limit { limit: u64, offset: u64 },   // unchanged
    /// Passes rows through unchanged, at no cost.
    PassThrough,   // unchanged
    /// PromQL `offset`: moves when the input is evaluated.
    TimeShift,                       // new: free; rows pass unchanged
    /// Keeps the input's rows inside a time window.
    TimeRange,                       // new: priced by the rows it keeps
    /// PromQL range selection over `range_millis`.
    PromqlRange { range_millis: u64 },   // unchanged
    /// PromQL subquery over `range_millis` at `resolution_millis`.
    PromqlSubquery { range_millis: u64, resolution_millis: Option<u64> },   // unchanged
    /// PromQL binary operation between vectors or scalars.
    PromqlBinary { operation: PromqlBinaryOperation, operand_mode: PromqlBinaryOperandMode,
        cardinality: PromqlVectorCardinality, build_side: Option<HashJoinBuildSide> },   // unchanged
    /// PromQL label rewriting.
    PromqlRelabel { expression_operations_per_row: u64 },   // unchanged
    /// PromQL info-metric enrichment.
    PromqlInfoEnrich { matcher_operations_per_info_row: u64 },   // unchanged
    /// PromQL `limitk` / `limit_ratio` series sampling.
    PromqlSeriesSample { kind: PromqlSeriesSampleKind, grouping_key_count: u64 },   // unchanged
    /// PromQL `vector(scalar)`.
    PromqlScalarToVector,   // unchanged
    /// PromQL `scalar(vector)`.
    PromqlVectorToScalar,   // unchanged
    /// A PromQL scalar literal.
    PromqlScalarLeaf,   // unchanged
    /// PromQL per-series range function.
    PromqlPerSeries { operations_per_row: u64, accumulator_count: u64 },   // unchanged
    /// PromQL `absent`, `absent_over_time` and `present_over_time`.
    PromqlPresence { kind: PromqlPresenceKind, operations_per_row: u64 },   // unchanged
}

crates/asap-physical-operators/src/summary_kernels/exact.rs

/// Planned evaluation of an exact summary. The type stays; `offset_ms` is new.
pub struct ExactEvaluation {
    /// The statistic to read.
    pub statistic: Statistic,        // unchanged
    /// The logical PromQL counter window, if any; default `None`.
    #[serde(default, skip_serializing_if = "Option::is_none")]
    pub lookback_ms: Option<i64>,    // unchanged
    /// How long before the evaluation time the window ends; default 0.
    #[serde(default, skip_serializing_if = "is_zero")]
    pub offset_ms: i64,              // new: PromQL offset; struct literals must set it
}

Usage, end to end (checked by running the planning stages at 3466f32f with a scratch test, and stage_pipeline --example planner-layering-3b): Example 3, Pattern B.

// What the user writes (`planner_layering_examples::pattern_b()`): `entry` is a p99
// panel over the last 5 min, evaluated every minute, rank ε = 0.01, δ = 0.01, at most
// 200 ms. Its 1-min interval is what lets Stage 1 build the 5-min window as 1-min panes.
let entry = RepeatingEntry {
    query: Query("quantile_over_time(0.99, latency_ms[5m])".into()),
    demand: RepeatedDemand::FixedInterval(RepetitionInterval(60_000)),
    requirements: QueryRequirements {
        accuracy: AccuracyRequirement::Explicit(
            RequestedAccuracy::epsilon_delta(0.01, 0.01).with_metric(ErrorMetric::Rank),
        ),
        response_latency: LatencyRequirement::ExplicitMaxMs(200.0),
    },
    predictability: Predictability::Predictable { known_at: None },
    time_selection: TimeSelection {
        scope: QueryTimeScope::RealTime,
        lookback: Some(DurationMs(300_000)),
        as_of: None,
    },
};
// `workload` is the planner input: the entry plus the shared data facts
// (1M series, a sample every 15 s, continuously ingesting).
let workload = PlanningWorkload {
    query_workload: QueryWorkload {
        language: QueryLanguage::PromQL,
        query_batch: None,
        repeating_queries: Some(vec![entry]),
    },
    data_workload: Some(shared_data_workload(DataArrival::ContinuouslyIngesting)),
};
// `roots` holds the PromQL frontend's lowered query; the stages plan over it.
let roots = lower_promql_roots(&workload)?;
// `run` holds every stage's output. Stage 1 runs Pass 1 (L1 exact, L2 KLL, L4 DDSketch),
// then, for each Pass 1 candidate, the window-composition rule below, which adds
// L3 (KLL in 5 × 1m panes) and L5 (DDSketch in 5 × 1m panes).
let run = run_planning_stages(&workload, roots, None)?;

// Inside Stage 1, for L2 (the whole-window KLL):
// `l2` holds L2's query roots: one KLL SummaryAgg over TimeRange(5m), then the quantile.
let l2 = run.stage1_logical_asap_optimization[1].roots.as_ref()?;
// `sites` holds the builds that can be split into panes: here one, the KLL, with
// pane_width_ms = 60,000 (gcd of the 5-min window and the 1-min interval),
// pane_count = 5, readers = [0]. `&[Some(60_000)]` is each query's repeat interval.
let sites = pane_sites(l2, &[Some(60_000)]);
// `variants` holds one candidate per non-empty subset of `sites`: here one, the KLL
// built as a PaneChain of 5 panes under a SummaryMerge. It becomes L3.
let variants = window_composition_variants(l2, &sites)?;

// What the user gets: `selection` is Stage 3's choice among P1–P5.
let selection = run.stage3_plan_selection?;
assert_eq!(selection.selected, "P2");   // whole-window KLL, 2.100 cost_per_second; k enlarged to 613 for δ = 1e-8 per series
// P3 (KLL panes): valid, 2.183, one build per minute plus 1M × 5 pane merges. P1 (exact): 2.400.
// P4 and P5 (DDSketch, whole or in panes): invalid, a relative bound misses the rank target.

// Had P3 been selected, the executor would run its root:
// `planned` holds L3's query root, the KLL still a PaneChain under a SummaryMerge.
let QueryRoot::Operator(planned) = variants[0].roots.as_ref()?[0].clone() else { unreachable!() };
// `expanded` is the same root with the chain replaced by its 5 pane SummaryAggs
// (no PaneChain left); the executor compiles this, building each pane and merging them.
let expanded = expand_pane_chains(&planned)?;

Before this PR

  • The planner never emitted panes; pane plans existed only hand-written in tests.
  • A TimeShift was charged one operation per row.
  • A scan under a TimeShift was priced as one minute of data (the default lookback) whatever the window.
  • A TimeRange passed every row of a shared scan through.
  • A counter evaluation's window was its own TimeRange, ending at the evaluation time: one pane wide over panes, and ignoring offset.

After this PR

Before After
Example 3b candidates 3: exact, KLL, DDSketch 5: adds KLL and DDSketch in five 1-min panes
Example 3b selected KLL, 2.100/s KLL, 2.100/s; KLL panes 2.183/s (whole build + 1M × 5 merge ops per minute); exact 2.400/s; DDSketch whole or in panes misses the rank target
Example 1 candidates 9 27: Q1's rate and Q2's option (exact sum, CMS+heap, CountSketch+heap) in six 10-s panes, each alone or both, with or without a shared scan
Example 1 selected P1 all exact, 7.940/s P1, unchanged; pane plans cost 8.540/s or more, heap pane plans are invalid like their whole builds
Example 3a P485 all KLL, 21,724.8/s P485, 39,828.8/s: offset queries' scans now read their 1–3 years instead of one minute; all-exact is still ~16% more (46,252.8/s). No panes (run once).
Example 2 82 candidates unchanged (SQL windows are not a TimeRange)
L3 Q1 Kll · Kll for Q1 in 5 × 1m panes
  SummaryEstimate(p99) ← SummaryMerge ← PaneChain{Kll k=269, 60000 ms × 5} ← Scan

At query time the panes never win: they read the same rows as the whole build and then merge. They pay off only when kept between evaluations (Stage 2 materialization, G8).

Verification

Run on head 07a86fbd, rebased on main a18fa4a6:

  • cargo nextest run -p asap-types -p asap-aware-mapping -p asap-physical-operators -p asap-integration-tests -p asap-devtools: 1329 passed, 31 skipped.
  • cargo fmt --all -- --check: passed.
  • cargo clippy --workspace --all-targets --all-features --locked -- -D warnings: passed.
  • tools/verify_metricsql_external_consumer.sh: passed.

New and changed tests:

  • types/tests/pane_chain.rs:

    • a merged chain covers (-5m, 0] like the whole build;
    • pane i covers (-(o+(i+1)w), -(o+iw)] with the offset folded;
    • expansion gives five panes with the same coverage;
    • a chain over a range vector, or with no panes, is invalid.
  • asap_aware_cse unit tests:

    • width = gcd: 2 min → 5 × 1m, 90 s → 10 × 30 s, no panes at 5 min or without an interval;
    • Rate, Increase, Sum and Max accumulators get panes, IRate does not;
    • the variant reads SummaryMerge(PaneChain) with the build's family, schema, coverage and guarantee;
    • two sites give three subsets.
  • Pricing:

    • analytical_cost: TimeShift is free; TimeRange costs what it keeps and cannot keep more than it reads.
    • stage3_plan_selection::time_shift_is_free_and_the_scan_reads_the_shifted_window was written first and failed on 2c674f69: the shift cost 0.01.
  • tumbling_pane_merge.rs: for sum_over_time, quantile_over_time, increase and rate, the planner's chain:

    • keeps the whole build's guarantee;
    • expanded, returns exactly the single build's rows;
    • unexpanded, is rejected by compile.

    Review-fix tests, each written first and failing before the fix:

    • exact_rate_over_tumbling_panes_matches_single_build: 0.0667 per series;
    • exact_increase_off_the_pane_boundaries_matches_single_build: samples 7 s off the pane boundaries;
    • offset_counter_window_ends_at_the_offset: rate/increase(m[5m] offset 1m) one minute later equals the plain query.
  • Example 3:

    • b_window_composition_adds_tumbling_per_mergeable_option: the 5 candidates, 1-min × 5 panes, a merge only under panes;
    • b_panes_rebuilt_at_query_time_add_the_merge: 2.100 vs 2.183;
    • b_window_parameters_are_legal is no longer vacuous;
    • window_form reads a PaneChain as Tumbling.
  • Example 1: Pass 1 and identical-expression tests read the whole-window candidates. New tests cover the 10-s panes (27 candidates) and pane plans costing more or being invalid.

  • Devtools: Example 1 counts 9 → 27, and the committed Example 1 fixture is regenerated (0.6 → 1.8 MB).

Evidence types: no screenshots (no UI change beyond the viewer's PaneChain kind category) and no measurements; the costs above are the illustrative model's.

Limitations and follow-ups

…ummary

A PaneChain is the SummaryAgg of a family built over each of pane_count
back-to-back range windows of pane_width_ms, newest first, over the input
a range selector would read. A SummaryMerge over the chain is the state of
one build over the whole window: the chain's coverage is that build's, and
schemas, timing and the readers' schema derivation look through the merge.

`ir::pane_chain` gives the equivalences: `whole_window_build`, `pane(i)`
(a range over the input shifted back i panes, folded into an existing
offset) and `expand_pane_chains`, which rewrites each merge over a chain
into a merge over its explicit panes, the shape the executor already runs.
The executor rejects an unexpanded chain with that instruction.
…mple 3)

Pass 2's window-composition rule, for one query's tumbling panes: a
SummaryAgg over a range TimeRange(W), whose family merges in the
family-capability table (exact accumulators included; heap top-k merges
only approximately), read by queries repeating at a fixed interval E, can
be built as W / w panes of w = gcd(W, E). Every non-empty subset of such
builds is a candidate in which each becomes a SummaryMerge over one
PaneChain, enumerated lazily after the identical-expression and
summary-capability candidates, each followed by its own
identical-expression candidates.

A read of merged panes gets the guarantee of the same read of one build
over the whole window when the family's merged error is exact or the same
as that build's, and none when it is approximate.

Stage 3 prices a chain as one build over its whole window plus one
operation per row to place it, and a merge at one operation per state.
The generic per-operator pricing now makes a TimeShift free and prices a
TimeRange by the rows it keeps; a query-time scan reads the longest
window above it through any time shift.

Example 1 has 27 Stage 1 candidates (Q1's rate and Q2's option in six
10-s panes); Example 3b has 5 (KLL and DDSketch in five 1-min panes).
Selections are unchanged: panes rebuilt at query time cost the whole
build plus the merge.
The Example 3 acceptance spec and the stage viewer's Example 1 and 3b rows
describe the tumbling-pane candidates and their costs.
…shifted ranges

A Rate or Increase evaluation took its lookback from the TimeRange it
reads and ended it at the evaluation time. Over expanded tumbling panes
each pane's range is one pane wide, so rate(m[5m]) divided the merged
increase by 60 s instead of 300 s (0.333 vs 0.0667), and Increase
extrapolated to the newest pane's start. Under `offset`, even one build
ended its window at the evaluation time instead of before the offset.

The counter window is now the union of the range windows the evaluation
reads, each moved back by the time shifts below its range: its lookback
and its offset (`ExactEvaluation::offset_ms`) come from that union, and
ranges that do not form one window are rejected as before.
The Example 1 acceptance spec counts the 27 candidates with tumbling
panes. `expand_pane_chains` documents that a chain's materialization is
dropped and left to Stage 2.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CSE enhancement New feature or request window Window functions, frames, panes, or temporal semantics

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant