Repository navigation
[Design] Why the SummaryExpr is designed like this? #463
Description
See the following code
fn summary_children(expr: &SummaryExpr) -> Vec<&Rc<SummaryNode>> {
match expr {
SummaryExpr::KeepPreAsap(_) => vec![],
SummaryExpr::BinaryOp { lhs, rhs, .. } => vec![lhs, rhs],
SummaryExpr::CandidateTopK { candidates, values, .. } => vec![candidates, values],
SummaryExpr::ValueOperation { child, .. } => vec![child],
SummaryExpr::RelationalJoin { left, right, .. } => vec![left, right],
SummaryExpr::SummaryAgg { child, .. } => vec![child],
SummaryExpr::SummaryJoin { outer, inner, .. } => vec![outer, inner],
SummaryExpr::SummarySubtract { left, right } => vec![left, right],
SummaryExpr::SummaryDelete { summary_input, .. } => vec![summary_input],
SummaryExpr::SummaryEstimate { summary_input, .. } => vec![summary_input],
SummaryExpr::SummaryMerge { children } => children.iter().collect(),
}
}
I don't know what is the design principle of these branches. For example, I cannot understand
- Why
ValueOperationis a parallel concept withBinaryOp:BinaryOplooks like a special case of projection, which is a sub-concept ofValueOperation. - Why
KeepPreASAPis a parallel concept withValueOperation: They both contain lots of repeated concepts, like projection, filter, so on and so forth - What is the relation between
ExactAggregateinSummaryAggand the original aggregation inKeepPreAsap?
I feel there are a lot of semantic overlaps in the post-ASAP-DAG language and am really lost.
Activity
zzylol commentedon Sep 24, 2026
The confusion is understandable:
SummaryExprmixes several classification axes, rather than representing a strict hierarchy of operator semantics. The three distinctions in the current implementation are:1.
BinaryOpvs.ValueOperationValueOperationwraps one post-ASAP child, whileBinaryOpconnects two independently planned children and carries PromQL vector-matching semantics.Arithmetic between columns of a single input row can be represented as a projection. Arithmetic between independently computed vectors also needs to match elements across the two inputs. The separate variant preserves both input plans and those matching semantics.
So the distinction is primarily about input structure and composition. The name
ValueOperationis broader than its current single-child representation, which makes the overlap confusing.2.
KeepPreAsapvs.ValueOperationThese represent different planning boundaries:
KeepPreAsapembeds an entire originalQueryExprsubtree. Its internal operators are not exposed as post-ASAP nodes.ValueOperationexposes an operation over a post-ASAP child, allowing that child to contain independently selected summaries and readouts.For example:
retains the original computation as a whole, whereas:
allows the aggregate to use a summary while preserving the projection above its readout.
The projection/filter semantics can therefore be the same in both representations. What differs is how much of the computation has been exposed to post-ASAP planning. This also explains why
summary_children(KeepPreAsap(...))returns no children: it has noSummaryNodechildren, even though it contains aQueryExprsubtree.3.
ExactAggregateinSummaryAggvs. an original aggregate inKeepPreAsapBoth can produce exact results, but they expose different execution contracts:
KeepPreAsap(Aggregate(...))SummaryAggwith anExactAggregatefamilyValueOperation(Exact(Aggregate(...)))Here, “summary” does not necessarily mean “approximate.” An exact accumulator is also summary state, with explicit maintenance and applicable merge behavior.
The current implementation uses
FinalizeExactAccumulatorto mark the transition from maintained exact state to a read-time value, even when the accumulator state already represents the scalar result. Approximate summaries instead useSummaryEstimatefor their readout.The organizing distinctions are therefore retained logical subtrees, computation over ordinary values, and computation over summary state, with input structure and execution timing providing additional dimensions. The semantic overlap is real, but these variants expose different planning and execution boundaries.
References: expression definitions, execution-state rules.
Selvomega commentedon Sep 24, 2026
Further questions:
Against your response 1:
Is it true that BinaryOp is only used for PromQL, and has nothing to do with the SQL world?
Against your response 2:
KeepPreASAPnot re-shaped by the optimizer at all? I see some generated plans half-ValueOperationand hald-KeePreASAP,Projectiononly semantically mean that I am going to do some per-row transformation from my input data. Why you think pre-ASAPProjectionand post-ASAPProjectionneed to be distinguished? (since to me they have the same semantic)Against your response 3:
The way I understand your explanation is: Our "Summary" actually means "State". And in our current design, we are considering the query execution in the following perspectives:
Summarydefined by us v.s. othersBased on the above dimensions, we have
KeepPreAsap(Aggregate(...))fall intoothers + agg + legacySummaryAggfall intosummary + agg + asapValueOperation(Exact(Aggregate(...)))falls intoothers + agg + asapIs this understanding correct?
In general
As you yourself said: "
SummaryExprmixes several classification axes", can you articulate what are the classification axes?I tried to mentally reverse-engineer some axes in the section above, but I feel it would be better to have you articulate the design intention directly.
Also, according to what I have reversed out, I feel some axes may be unnecessary, like "legacy execution vs ASAP execution". However, now I am not sure either if this an actual axis, or if my feeling of un-necessity is correct.
Therefore, I feel it will be extremely helpful for you to articulate out the design considerations of the post-ASAP dag language. :)