feat(logical-asap-optimization): Example 2 — summary-capability rule and UnivMon frequency statistics

[zzylol]

Why

#509 Example 2 asks for one summary serving several computations: one UnivMon over src_ip feeding the distinct count, the entropy and the L2 norm of the per-source counts. That needs Pass 2's summary-capability rule, a table of which readouts each family answers, executor support for the exact and UnivMon options of the three statistics, and SQL recognition of the entropy and L2 idioms. This PR consolidates that work (G5 of the upper-stack consolidation).

What

• Family-capability table (types). FamilyCapability (from feat(types): family-capability table and executor tumbling pane merges #696) now also lists the readouts one state of each family answers (Readout: total, item frequency, quantile, distinct, L2, entropy, top-k), via FamilyCapability::answers. UnivMon answers its total and every frequency moment, KLL/DDSketch every rank, CountSketch item frequencies and L2, the heap sketches top-k and item frequencies. Exact accumulators answer no readout.

• Pass 2 summary-capability rule (logical ASAP optimization). Builds of one candidate that chose the same family over the same input sub-DAG (window, WHERE and time range included), item, weight, grouping and filter, whose every reader is a readout the family answers, form a group. For every subset of two or more members whose sizes differ (partial sharing), the rule adds a candidate in which that subset is one build. The build is sized with each parameter's largest value among the members; KLL's failure probability, DDSketch's α and UnivMon's recorded distinct-count failure probability take the smallest value. Every guarantee above it is composed again, so plan selection checks each consumer against the shared state. Builds of equal size stay with the identical-expression rule. Unshared builds are untouched, and no consumer changes family.

• SQL frontend. SQRT(SUM(c * c)) over Float64 counts (CAST(c AS DOUBLE) or CAST(COUNT(*) AS DOUBLE)) lowers to FrequencyL2. -SUM(p * LN(p)) with p = COUNT(*) / SUM(COUNT(*)) OVER () lowers to FrequencyEntropy. Both keep the original output schema and decide SQL's empty-input NULL from an exact population count joined beside the statistic. The Int64 SUM(c * c) stays relational, because integer overflow is observable.

• Executor.

  • Exact Cardinality, FrequencyL2 and FrequencyEntropy reducers, memory-accounted and cooperative.
  • UnivMon is now a native family. One state answers the total, distinct count, L2 and entropy, over Bool, Int64 and Utf8 identities as type-tagged key bytes.
  • A unit-frequency update over one item column binds for UnivMon and HLL.
  • UnivMon's memory charge bounds the retained heap keys by the smaller of two bounds: every slot holding the longest key seen, and every layer holding a copy of every key inserted. It is constant time. asap_sketchlib has no eviction hook, so a stream of long keys is still overcounted. A build charges memory after every row, so walking the heaps made ingestion quadratic in the heap: 3.3 s for a 20,000-key heap on 19 layers over 40,000 keys, in release.

• Guarantees through SQL's NULL decision. SQL's L2 and entropy outputs decide the empty-input NULL from an exact population count cross-joined beside the estimate, through a CASE on that count. Two narrow rules now cover that shape:

  • A join (inner, outer or cross) keeps the approximate input's guarantee when every other input is exact and its predicate reads only exact columns. A key on the approximate input's columns, a semi- or anti-join, or two approximate inputs have none.

  • A CASE whose operand and conditions are exact keeps its weakest branch's guarantee; a literal or NULL branch is exact. A condition over an approximate value has none.

  • A projection over a join (directly or through row-preserving operators) reads each column's own guarantee, not the child's sample value. Plan selection's operand-domain lookup does the same. A projection over a computed projection over a join has no guarantee.

  • These rules assume every approximate node of Stage 1 has an exact row set: top-k estimates, whose rows are approximate, have no bound.

  Guarantees are per node, so every column of the approximate input counts as approximate. Column-level tracking (Track exact vs approximate per column so key-only filters over approximate inputs keep their guarantee #689) stays the general solution.

• Root metric through SQL's NULL guard (Accuracy targets and guarantees: three error metrics (absolute, relative, rank), a shared AccuracySpec, and metric-aware satisfies #690 API). A root's default metric now looks through a CASE whose value branches read columns or (cast) literals. It follows those columns through joins and row-preserving operators to the aggregate's measure, so SQL Q3 resolves to a relative target like the PromQL L2 root. Any other expression stays absolute.

• Ported to Accuracy targets and guarantees: three error metrics (absolute, relative, rank), a shared AccuracySpec, and metric-aware satisfies #690/feat(types): three error metrics, a shared AccuracySpec and metric-aware satisfies (#690) #708.

  • SQL L2 intents carry a relative target, entropy intents an absolute one, and the NULL-guard count an exact target.
  • Tests use RequestedAccuracy and AccuracyTarget::exact().

• Devtools. A stage_pipeline --example planner-layering-2 example. The stage-viewer tab 2 row is regenerated from it: it said 64 plans with 8 valid; it now lists 82 plans with 12 valid.

How

• asap_aware_cse::capability_groups / summary_capability_variants enumerate the subsets (mixed-radix over groups, binary-counter subsets within a group).
• estimators::shared_size_params computes the per-parameter shared size. Not every bound is monotone in its parameters: a Count Sketch's collision bound is tighter at a power-of-two width than one column wider. So the shared state is not assumed to meet each target; every guarantee above it is composed again and plan selection checks it. It returns no shared UnivMon when the combined rows and columns exceed asap_sketchlib's hash budget (rows · (⌈log2 cols⌉ + 1) ≤ 128, from feat(planner): certify UnivMon L2 and distinct count on the certifiable sketchlib estimators #702), since the library would not build that shape.
• A group of more than 20 builds is an error (MAX_GROUP_MEMBERS): every subset is a candidate, about a million at 20. A variant whose guarantee composition fails is listed as a candidate that cannot be composed, and the other variants go on.
• Stage 1 lists each composed candidate, its identical-expression candidates, then each summary-capability candidate followed by its own identical-expression candidates, every distinct DAG once.
• The Pass 2 record (Pass2Candidate) names the shared summaries, e.g. "UnivMon for Q1, Q3".
• UnivMon sizing and the L2 and distinct-count guarantees come from feat(planner): certify UnivMon L2 and distinct count on the certifiable sketchlib estimators #702 (estimators/univmon.rs); this PR reuses them and adds no certification of its own.

Key code interfaces added / modified

crates/types/src/post_asap/family_capability.rs, post_asap/mod.rs

/// What one summary family can do: its merge, subtraction and readouts.
/// The type stays (from #696); it gains one field.
pub struct FamilyCapability {
    /// How the family's states merge; `None` when there is no sound merge.
    pub merge: Option<MergeCapability>,   // unchanged
    /// Whether a state subtracts out of another of the same family (reserved).
    pub subtractable: bool,   // unchanged
    /// The readouts one state answers; empty for exact accumulators (read only
    /// as a finalized value) and unlisted families.
    pub readouts: &'static [Readout],   // new: what one state answers, whatever its parameters
}

/// One kind of `SummaryEstimate` readout, for the capability table. New type.
pub enum Readout {
    /// The total of the updates: a bare `PointCount` (no item value).
    Total,   // new
    /// One item's frequency: a `PointCount` with an item value.
    ItemFrequency,   // new
    /// The value at any quantile rank.
    Quantile,   // new
    /// The number of distinct items.
    Distinct,   // new
    /// The L2 norm of the item frequencies.
    FrequencyL2,   // new
    /// The entropy of the item frequencies.
    FrequencyEntropy,   // new
    /// The heaviest items, any `k` up to the state's heap.
    TopK,   // new
}

impl Readout {
    /// The kind of readout `statistic` is.
    pub fn of(statistic: &SketchStatistic) -> Self;   // new
}
impl FamilyCapability {
    /// Whether one state of the family answers `statistic`, whatever its parameters.
    pub fn answers(&self, statistic: &SketchStatistic) -> bool;   // new
}

/// Re-exports the table's types at `post_asap::`.
pub use family_capability::{FamilyCapability, MergeCapability, MergedError, Readout, SelectionRelation};  // + Readout

crates/asap-aware-mapping/src/asap_aware_cse.rs

// new: the Pass 2 summary-capability rule
/// Summary builds of one family over the same input data that differ only in
/// size, in children-first order; the unit the rule shares within. New type.
pub struct CapabilityGroup {
    /// The family every member builds.
    pub family: SketchAlgorithm,   // new
    /// The group's summary build nodes.
    pub members: Vec<Rc<OperatorNode>>,   // new
}
/// One shared build of a variant: members of one group that read one state. New type.
pub struct SharedSummary {
    /// Index of the group in the `capability_groups` result.
    pub group: usize,   // new
    /// Indices into that group's `members`.
    pub members: Vec<usize>,   // new
}
/// One Pass 2 candidate of the rule: the roots with each shared set made one
/// build sized for all its readers, guarantees composed again. New type.
pub struct SummaryCapabilityVariant {
    /// The builds this variant shares.
    pub shared: Vec<SharedSummary>,   // new
    /// The rewritten roots, or why composing their guarantees failed (affects
    /// only this variant).
    pub roots: Result<Vec<QueryRoot>, PerComputationCandidateError>,   // new
}

/// The most builds one group may have: every subset of two or more is a
/// candidate, about a million at 20.
pub const MAX_GROUP_MEMBERS: usize = 20;
/// The groups of at least two builds whose sizes are not all equal.
pub fn capability_groups(roots: &[QueryRoot]) -> Vec<CapabilityGroup>;
/// Every variant: per group, no sharing or one subset of two or more members.
/// Errors only for a group over `MAX_GROUP_MEMBERS` (too many subsets to enumerate).
pub fn summary_capability_variants(
    roots: &[QueryRoot],
    groups: &[CapabilityGroup],
) -> Result<Vec<SummaryCapabilityVariant>, PerComputationCandidateError>;  // one per choice of shared builds

crates/asap-aware-mapping/src/pass/stage1_to_stage3.rs, pass/mod.rs

/// A Stage 1 candidate Pass 2 added to the run's output, derived from one Pass 1 candidate.
/// The type stays, renamed from `IdenticalExpressionCandidate`; it covers both Pass 2 rules.
pub struct Pass2Candidate {
    /// The candidate's display id.
    pub id: String,   // unchanged
    /// The id of the Pass 1 candidate it derives from.
    pub from_logical: String,   // unchanged
    /// Summary build sets made one build each (summary-capability rule).
    pub summaries: Vec<String>,   // new
    /// Identical node groups made one node (identical-expression rule).
    pub shared: Vec<String>,   // unchanged
}

/// The output of Stages 1 to 3 for one workload. The type stays; one field is renamed.
pub struct Stage1ToStage3Run<Id> {
    /// Stage 1, Pass 1: the per-computation candidates, with the roots they read.
    pub stage1_pass1_per_computation: PerComputationCandidates<Id>,   // unchanged
    /// Stage 1, Pass 1: the number of composed whole-workload candidates (saturating).
    pub combinations: usize,   // unchanged
    /// The candidates both Pass 2 rules added, in enumeration order.
    pub stage1_pass2: Vec<Pass2Candidate>,   // was stage1_pass2_identical_expression: Vec<IdenticalExpressionCandidate>
    /// Stage 1: every whole-workload logical candidate, each Pass 1 one followed by its Pass 2 ones.
    pub stage1_logical_asap_optimization: Vec<LogicalCandidate>,   // unchanged
    /// Stage 2: one physical candidate per Stage 1 candidate that composed and was implemented.
    pub stage2_physical_asap_optimization: Vec<PhysicalCandidate>,   // unchanged
    /// Stage 2: the Stage 1 candidates Stage 2 could not implement, with the reasons.
    pub stage2_pruned: Vec<PrunedCandidate>,   // unchanged
    /// Stage 3: plan selection over the Stage 2 candidates.
    pub stage3_plan_selection: Result<Selection, SelectionError>,   // unchanged
    /// Stage 3: the selected candidate's logical roots with its confirmed weight requirements.
    pub stage3_selected_logical: Option<Vec<QueryRoot>>,   // unchanged
}

/// One Stage 1 candidate as plan selection sees it. The type stays; `pass2_of` changes type.
pub struct Stage1Candidate {
    /// Human-readable name, used only for display.
    pub display_name: String,   // unchanged
    /// The Pass 1 choice (`targets[t].choices.candidates[choice[t]]`); a Pass 2 candidate has its Pass 1 candidate's.
    pub choice: Vec<usize>,   // unchanged
    /// The candidate's roots, or why composing them failed.
    pub roots: Result<Vec<QueryRoot>, PerComputationCandidateError>,   // unchanged
    /// For a Pass 2 candidate, what it shares of which Pass 1 candidate; `None` for a Pass 1 candidate.
    pub pass2_of: Option<Pass2Of>,   // was Option<(usize, Vec<String>)>
}
/// What a Pass 2 candidate shares of a Pass 1 candidate. New type.
pub struct Pass2Of {
    /// The Pass 1 candidate's position in the enumeration.
    pub from: usize,   // new
    /// The summary builds made one build each.
    pub summaries: Vec<String>,   // new
    /// The identical groups shared (operator kinds, children first).
    pub shared: Vec<String>,   // new
}

crates/asap-aware-mapping/src/per_computation_candidates.rs, accuracy/estimators/mod.rs

// new, crate-internal
/// Pass 2 calls it after re-sizing a shared summary; a sub-DAG shared
/// between roots stays shared.
/// Re-derives guarantees through joins, projections and CASE after sharing.
pub(crate) fn with_composed_guarantees(
    roots: &[QueryRoot],
) -> Result<Vec<QueryRoot>, PerComputationCandidateError>;

/// `None` for different algorithms or a UnivMon the library would not build;
/// each reader's target is checked again later.
/// Largest value of each parameter among a shared build's readers.
pub(crate) fn shared_size_params(params: &[&SketchParams]) -> Option<SketchParams>;

crates/asap-physical-operators/src/operators/aggregate/mod.rs

/// An exact relational aggregate. The type stays; it gains three variants.
pub enum Reduction {
    /// Row count.
    Count,   // unchanged
    /// Exact distinct count of the tuples of these columns; NULL-containing tuples skipped.
    Cardinality(Vec<usize>),   // new
    /// Exact L2 norm of one identity column's unit-update frequencies; NULL identities skipped.
    FrequencyL2(usize),   // new
    /// Exact Shannon entropy in bits of one identity column's frequencies; NULLs skipped.
    FrequencyEntropy(usize),   // new
    /// Sum of one column.
    Sum(usize),   // unchanged
    /// Mean of one column.
    Avg(usize),   // unchanged
    /// Minimum of one column.
    Min(usize),   // unchanged
    /// Maximum of one column.
    Max(usize),   // unchanged
    /// PromQL `quantile` of one column: linear interpolation between closest ranks.
    Quantile { column: usize, q: f64 },   // unchanged
}

crates/asap-physical-operators/src/summary_kernels/factory.rs, univmon.rs

/// Shared update interface for query-time and precompute-time accumulation.
/// The trait stays; it gains one method with a default.
pub trait AccumulatorUpdater: Send {
    /// Rejects a precompute input outside the representable domain (default: non-finite).
    fn validate_single_input(&self, value: f64) -> Result<(), String>;   // unchanged
    /// Update from one typed row value; the default accepts Float64 only, a
    /// frequency kernel may also take non-numeric identities.
    fn update_value(&mut self, value: &crate::values::Value, timestamp_ms: i64) -> Result<(), String>;   // new
    /// Feed one (value, timestamp) pair, for single-subpopulation kernels.
    fn update_single(&mut self, value: f64, timestamp_ms: i64);   // unchanged
    /// Feed one keyed (key, value, timestamp) triple, e.g. a frequency item.
    fn update_keyed(&mut self, key: &KeyByLabelValues, value: f64, timestamp_ms: i64);   // unchanged
    /// Extract the final accumulator.
    fn take_accumulator(&mut self) -> Box<dyn AggregateCore>;   // unchanged
    /// Non-destructive read of the current state; pane-based sliding windows read shared panes with it.
    fn snapshot_accumulator(&self) -> Box<dyn AggregateCore>;   // unchanged
    /// Consume the updater and return its accumulator by move (default: a snapshot).
    fn into_accumulator(self: Box<Self>) -> Box<dyn AggregateCore>;   // unchanged
    /// Reset internal state for reuse.
    fn reset(&mut self);   // unchanged
    /// Whether this updater consumes keyed updates.
    fn is_keyed(&self) -> bool;   // unchanged
    /// Estimated memory usage in bytes.
    fn memory_usage_bytes(&self) -> usize;   // unchanged
}
// UnivMonUpdater overrides update_value to take typed identities

/// UnivMon summary state.
impl UnivMonAccumulator {
    /// One occurrence of a typed identity; NULL is skipped.
    pub fn insert_value(&mut self, value: &crate::values::Value) -> Result<(), Error>;  // new: Bool, Int64, Utf8 keys stay distinct
}

crates/asap-physical-operators/src/capability.rs, values.rs

/// The one item column of a unit-count frequency update, which an unkeyed
/// frequency family (HLL, UnivMon) builds from; `None` for other updates.
pub(crate) fn unit_frequency_column(
    update: &planner_types::post_asap::SummaryUpdate,
) -> Option<&planner_types::pre_asap::ColumnRef>;   // new

/// Checks a stored state matches its declared family and parameters before it is used.
fn validate_state(family: &SummaryFamilyType, state: &dyn AggregateCore) -> Result<(), Error>;  // changed: checks UnivMon dimensions against SketchParams::UnivMon

crates/frontend-sql/src/frequency.rs, lib.rs

/// Recognition of the SQL frequency idioms (L2, entropy) at the query root.
pub mod frequency;   // new
/// `None` when neither idiom applies; such queries stay relational.
/// Rewrites the SQL entropy and L2 idioms into frequency intents; lower_sql_dialect now calls it.
pub fn recognize_frequency_idioms(root: &Rc<OperatorNode>) -> Option<Rc<OperatorNode>>;

crates/devtools/src/planner_layering_examples.rs, planning_stages.rs, bin/stage_pipeline.rs

// new: #509 Example 2
/// Example 2's three SQL queries over the last minute of `flows`.
pub const EXAMPLE2_SQL: [&str; 3] = [
    "SELECT COUNT(DISTINCT src_ip) FROM flows WHERE ts >= now() - INTERVAL '1 minute'",
    "SELECT -SUM(p * LN(p)) FROM (SELECT COUNT(*) * 1.0 / SUM(COUNT(*)) OVER () AS p FROM flows \
     WHERE ts >= now() - INTERVAL '1 minute' GROUP BY src_ip)",
    "SELECT SQRT(SUM(CAST(c AS DOUBLE) * CAST(c AS DOUBLE))) FROM (SELECT src_ip, COUNT(*) AS c FROM flows \
     WHERE ts >= now() - INTERVAL '1 minute' GROUP BY src_ip)",
];
/// (ε, δ) of Example 2's Q1, Q2 and Q3.
pub const EXAMPLE2_ACCURACY: [(f64, f64); 3] = [(0.02, 0.01), (0.05, 0.01), (0.01, 0.01)];
/// Example 2's `flows(ts, src_ip)` table.
pub fn example2_catalog() -> asap_frontend_sql::SqlCatalog;
/// Example 2: three dashboard panels over one flow stream, every 10 s over the last minute.
pub fn example2() -> PlanningWorkload;

// changed: each entry lowered with its own accuracy target and the workload's SQL dialect
/// Lowers every SQL workload entry over `catalog`'s tables, in workload order.
pub fn lower_sql_roots(
    workload: &PlanningWorkload,
    catalog: &asap_frontend_sql::SqlCatalog,
) -> Result<Vec<QueryRoot>, String>;

// Runs the stage pipeline on #509 Example 2.
// stage_pipeline: new CLI value --example planner-layering-2

Usage, end to end (numbers from stage_pipeline --example planner-layering-2 --out run.json at this PR's merge commit 2c674f69):

// 1. What the user writes: Example 2's three SQL panels over `flows(ts, src_ip)`, each every 10 s
//    over the last minute. `workload` holds them with their (ε, δ); no distinct bound is declared.
//      q1 SELECT COUNT(DISTINCT src_ip) ...                                   ε = 0.02, δ = 0.01
//      q2 SELECT -SUM(p * LN(p)) FROM (SELECT COUNT(*) * 1.0 / SUM(COUNT(*)) OVER () AS p ...)   ε = 0.05
//      q3 SELECT SQRT(SUM(CAST(c AS DOUBLE) * CAST(c AS DOUBLE))) FROM (... COUNT(*) AS c ...)   ε = 0.01
let workload = example2();

// 2. Stage 0 (query-language frontends): `roots` are the lowered queries. The SQL frontend now
//    recognizes q2 as FrequencyEntropy(src_ip) and q3 as FrequencyL2(src_ip), each beside an
//    exact row count that decides SQL's empty-input NULL. Pass 1 and Pass 2 read these roots.
let roots = lower_sql_roots(&workload, &example2_catalog())?;

// 3. Stage 1, Pass 1 offers each statistic exact or a summary and composes one candidate per
//    choice (16). `composed` is one of them: q1 UnivMon, q2 exact, q3 UnivMon, i.e. two
//    UnivMon builds over the same `src_ip` input, sized per readout (#702):
//      q1: UnivMon { heap_size: 337500, sketch_rows: 1, sketch_cols: 1, layers: 19, distinct_failure_probability: 0.01 }
//      q3: UnivMon { heap_size: 1, sketch_rows: 5, sketch_cols: 56817, layers: 1 }
// `per_computation` holds each query's Pass 1 options (q1: exact, HLL, KMV, UnivMon; q2 and q3:
// exact, UnivMon); Stage 1 enumerates its choices.
let per_computation = generate_per_computation_candidates(roots.into_iter().enumerate().collect())?;
// `composed` is that one choice's roots, the 52nd Stage 1 candidate in enumeration order
// (stage_pipeline's L52 "Q1 UnivMon · Q2 exact · Q3 UnivMon"); the Pass 2 rule below reads it.
let composed = stage1_logical_asap_optimization(&per_computation).nth(51).expect("L52").roots?;

// 4. Stage 1, Pass 2 summary-capability rule (new). `groups` holds the builds of `composed` that
//    one UnivMon state can serve (same family, input, item and grouping; FamilyCapability::answers
//    says UnivMon answers both the distinct count and L2). Here: one group, {q1's, q3's}.
let groups = capability_groups(&composed);
// Each `variant` makes one subset of a group a single build, sized with each parameter's largest
// value (shared_size_params), and recomposes every guarantee above it so Stage 3 checks it again.
for variant in summary_capability_variants(&composed, &groups)? {
    // `shared_roots` is the candidate's DAG; stage_pipeline lists it as
    // L55 "Q1 UnivMon · Q2 exact · Q3 UnivMon · one UnivMon for Q1, Q3", whose one build is
    //   UnivMon { heap_size: 337500, sketch_rows: 5, sketch_cols: 56817, layers: 19, distinct_failure_probability: 0.01 }
    let shared_roots = variant.roots?;
}

// 5. What the user gets back (stage3_plan_selection in run.json): 82 candidates, 12 valid.
//    selected: "P1" (all exact, 6.08 cost/s)
//    P4 (q3 through its own UnivMon, the rest exact): valid, 8.08 cost/s
//    P55 (the shared UnivMon for q1 and q3): rejected, "q1: the output guarantees a Relative bound
//      Some(0.02), failure probability None, ...": Example 2 declares no distinct bound, so the
//      distinct count's top-layer term stays unknown (with a bound it is certified, #702)
//    Any candidate reading q2 from a UnivMon: rejected, "q2: the output has no guarantee, ...":
//      UnivMon's entropy is not certified.

Before this PR

• Each query of Example 2 got its own UnivMon. No rule could share one UnivMon across the statistics.
• The executor rejected UnivMon builds ("summary family has no native DAG state implementation") and had no exact distinct, L2 or entropy reducer.
• SQL Q2 and Q3 were plain per-source counts.

After this PR

• On the Example 2 PromQL stand-in, Stage 1 offers one UnivMon for each pair of statistics and one for all three. The shared state is sized as the per-parameter maximum of the per-consumer sizes: Q1 is a 337,500-key heap on 19 layers, Q3 is a 5 × 56,817 Count Sketch, and the shared build is the larger of each.
• Every exact and UnivMon option of the three statistics executes, in PromQL and SQL.
• The SQL viewer example (planner-layering-2, regenerated row in docs/user_guide_docs/stage-viewer.md):
  • 82 plans: 16 from Pass 1, the rest from the identical-expression and summary-capability rules.
  • 12 are valid, including SQL Q3 through UnivMon (8.08 per second). P1 (all exact) is selected at 6.08 per second.

Verification

• cargo nextest run on asap-types, asap-aware-mapping, asap-frontend-sql, asap-frontend-promql, asap-physical-operators, asap-integration-tests and asap-devtools: all pass. The suite includes planner_layering_example2, sql_frequency, frequency_execution, sql_cardinality, univmon_execution, precompute_raw_samples and stage_pipeline. Three Example 2 tests stay #[ignore]d, each naming its missing feature: an entropy accuracy model, specialized entropy and L2 families, and materializing the shared UnivMon (Stage 2 materialization: ingestion vs query time per summary #604).
• New regression tests, each seen failing before its fix:
  • shared_size_params refuses a shared UnivMon past the hash budget.
  • UnivMon's memory charge does not grow with heap occupancy (memory_charge_does_not_grow_with_heap_occupancy). The timing check per_row_memory_charge_of_a_large_build_is_fast is ignored by default; run it in release. It took 3.3 s before the fix and passes well under its 0.5 s bound after.
  • A capability group of more than 20 builds is an error (a_group_past_20_members_is_an_error).
  • A projection over a join reads each column's guarantee (projection_over_a_join_reads_each_columns_guarantee): sum(m) * 1000 beside an exact column composes to absolute error 1000. It composed to 1 before the fix.
  • SQL's NULL guard keeps the statistic's root metric (root_metric_looks_through_a_null_guard_to_its_statistic).
  • One 1 KB key in a 20,000-slot, 19-layer UnivMon is charged at most once per layer (one_long_key_is_charged_about_once). It was charged 380 MB before the fix.
  • SQL Example 2 Q3 via UnivMon is accepted by plan selection (stage3_plan_selection_accepts_sql_univmon_l2_beside_its_exact_population_count).
  • Unit tests: a join beside an exact input on exact columns keeps the guarantee; a join on an approximate key, a semi-join, two approximate inputs, and a CASE testing an approximate column give none.
• cargo fmt --all -- --check and cargo clippy --workspace --all-targets --all-features --locked -- -D warnings: clean.
• Screenshots: not applicable (the viewer layout is unchanged; only the example and the doc row change).

Limitations and follow-ups

• Q1 UnivMon in the viewer example. It needs the source's distinct bound, which the example does not declare, so it shows "failure probability None". HLL is rejected for the same reason.
• Entropy stays uncertified (feat(planner): certify UnivMon L2 and distinct count on the certifiable sketchlib estimators #702), so every plan with UnivMon for Q2 is invalid.
• A shared UnivMon is charged once per consuming query while it is not materialized. Materialization comes with Stage 2 materialization: ingestion vs query time per summary #604 / G8.
• Equality ignores recorded failure probabilities. SketchParams equality ignores KLL's and UnivMon's recorded failure probabilities. Builds of equal shape but different δ therefore stay with the identical-expression rule, which is conservative: the state is the same.
• UnivMon has no top-k readout in the family-capability table, consistent with the executor, which does not evaluate one.
• No failing-first test for the per-variant composition error. No current plan makes a variant's composition fail, so that fix is structural: the per-variant Result makes it impossible for one failure to drop the others.
• Builds carry their sized parameters, not their targets. Pass 1 sizes each consumer alone, and Pass 2 takes the per-parameter maximum, so the shared state is the same as with target-carrying builds. This deviation was reviewed and accepted. Unlike the design text ("summary nodes are created unsized"), the identical-expression rule merges builds of equal size, not of equal target.
• The answers metric check from Stage 3: satisfies checks the error metric #627 is folded into Accuracy targets and guarantees: three error metrics (absolute, relative, rank), a shared AccuracySpec, and metric-aware satisfies #690. feat(executor): execute the exact SQL entropy fallback; Example 2 status #599's sql_window_sum / ln are left out. devtools: stage_pipeline examples planner-layering-2 and 4b #613's Example 4b is not part of this PR.

Supersedes

This PR supersedes #595, #596, #597, #598, #599, #621 and #613; they are closed unmerged. #621's UnivMon L2 certification already landed in #702.

[zzylol]

Planned simplifications of this PR's interfaces. They land in the Pass 2 *Candidate rename PR after #717, in addition to CapabilityGroup becoming CapabilitySharingCandidate and SummaryCapabilityVariant becoming SummaryCapabilityPass2Candidate:

• One Pass 2 record. Pass2Candidate and Pass2Of carry nearly the same fields, and those fields are ready-formatted display strings. They become one record, Stage1Candidate.pass2: Option<Pass2Candidate>, holding the Pass 1 candidate it came from plus the node ids it shares, rolls up, reuses or merges. The display text moves to devtools candidate_labels, as was already done for display_name.
• No index-based SharedSummary. SharedSummary { group: index, members: indices } makes the reader look things up in two other lists. The candidate holds the shared build nodes directly instead (shared_builds: Vec<Vec<Rc<OperatorNode>>>), and SharedSummary is deleted.
• FamilyCapability.subtractable is deleted. It is reserved, and nothing in the planner reads it.
• Readout becomes StatisticKind, so Readout::of(statistic) becomes statistic.kind(). FamilyCapability::answers(statistic) stays.
• Names that say what they do. with_composed_guarantees becomes recompose_guarantees, and shared_size_params becomes size_for_all_readers.
• One Example 2 function. EXAMPLE2_SQL, EXAMPLE2_ACCURACY, example2_catalog and example2 become example2() -> (PlanningWorkload, SqlCatalog). The accuracy targets are already in the workload.