Skip to content

feat(planner): export a planned workload as one physical ASAP DAG - #666

Merged
zzylol merged 5 commits into
mainfrom
stack/528-07c-planner-followups
Oct 13, 2026
Merged

zzylol merged 5 commits into
mainfrom
stack/528-07c-planner-followups

Conversation

@zzylol

@zzylol zzylol commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Why

With the planner on the unified IR (#542), a planned workload can be handed to the deployment runtime (Stage 4) as one physical DAG, and the summary coverage that #646 derives from the node can be checked end to end from real queries.

What

How

execution_timed_dag reuses #541's compile_physical_asap_workload_with_node_ids over the planned roots. The coverage tests read OperatorNode::coverage() on the KLL summary candidate; coverage is derived from the node, nothing is declared by the planner.

Before this PR

Each planned query could be exported only on its own, and coverage was checked only on hand-built nodes (#646).

After this PR

A planned batch exports as one PhysicalASAPDAG, and the coverage derivation is tested on SQL and PromQL queries through the frontends and Pass 1 candidate generation. New public API: PlanOutput::execution_timed_dag.

Evidence

From summary_coverage_e2e.rs: SQL region = 'us' → region ∈ {us}; latency < 100 → latency ∈ (−∞, 100); a renamed column (region AS r) → r ∈ {us}, with latency * 2 > 10 kept in the definition's scan; PromQL quantile_over_time(0.99, m[1m] offset 2m) → relative time (−3m, −2m]; quantile(0.99, rate(m{job="api"}[5m])) → unrestricted selection, with the matcher kept in the definition (it is below rate); quantile(0.99, rate(m[5m]) > 0) → value ∈ (0, ∞). From operator_design_examples.rs: a two-query batch exports with two roots and one shared SummaryAgg.

Verification

  • Coverage tests in summary_coverage_e2e.rs: sql_where_value_set_moves_to_selection, sql_where_range_moves_to_selection (exactly one box; the scan predicate is removed from the definition), sql_renamed_column_moves_and_expression_stays, promql_offset_window_is_relative_time, promql_matcher_below_rate_stays_in_definition, promql_comparison_above_rate_moves_to_selection, promql_vector_comparison_stays_in_definition (rate(a[5m]) > rate(b[5m]) is a BinaryOp, nothing selected) and promql_bool_comparison_stays_in_definition (> bool 0 is a Project, kept in the definition). The SQL tests also assert that the definition's SummaryAgg has no filter.
  • Workload export: batch_planning_replaces_and_shares_summary_operators (one physical DAG, two roots, shared state once); scalar_queries_read_planned_summaries exports scalar roots keyed by entry that reference the shared summary.
  • Other checks on the PR head: cargo fmt --check; workspace all-target Clippy with -D warnings; cargo test --workspace (1,734 passed, 0 failed, 2 ignored); dag-viewer test_render (31 passed); tools/verify_metricsql_external_consumer.sh OK.

Architectural decisions

Not applicable: both changes use existing interfaces.

Limitations and follow-up

Stack: tracker #528 · main → #542 → #665 → #666 → #543 → #667 → #561.

Human review — do not complete with an agent

  • The MVP boundary is correct.
  • New conceptual layers or public interfaces are necessary.
  • The before/after description matches the intended product behavior.
  • Human reviewer:
  • Decision and rationale:

🤖 Generated with Claude Code

https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

@zzylol
zzylol force-pushed the stack/528-07b-promote-roots branch from 7dc95a1 to 26b7ab0 Compare October 12, 2026 20:30
@zzylol
zzylol force-pushed the stack/528-07c-planner-followups branch from 3e0ec28 to fce55da Compare October 12, 2026 20:30
@zzylol
zzylol force-pushed the stack/528-07b-promote-roots branch from 26b7ab0 to 8aba6bc Compare October 12, 2026 22:55
@zzylol
zzylol force-pushed the stack/528-07c-planner-followups branch from fce55da to 1e719c9 Compare October 12, 2026 22:55
@zzylol
zzylol force-pushed the stack/528-07b-promote-roots branch from 8aba6bc to e6c4216 Compare October 13, 2026 01:43
@zzylol
zzylol force-pushed the stack/528-07c-planner-followups branch from 1e719c9 to 5c2bd85 Compare October 13, 2026 01:43
@zzylol
zzylol force-pushed the stack/528-07b-promote-roots branch from e6c4216 to b26bfdc Compare October 13, 2026 01:54
@zzylol
zzylol force-pushed the stack/528-07c-planner-followups branch from 5c2bd85 to 37e7726 Compare October 13, 2026 01:54
zzylol and others added 5 commits October 13, 2026 01:56
PlanOutput::execution_timed_dag times every query root, and every operator
reference of a standalone scalar root, with one shared TimingMemo and
compiles them with compile_physical_asap_workload, so a batch is one DAG
with a root per operator query, a scalar root per standalone scalar query
(keyed by entry) and shared sub-DAGs exported once. Lifecycle phases come
from the deployments of every plan, including the plans of scalar-read
sub-DAGs. The per-plan SummaryMaintenanceLifecyclePlan::execution_timed_dag
is the batch of one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
Lower each example query of the ASAP primitive schema design doc (#573
§4.2.2), plan it with the default cost model and check the coverage of the
selected KLL state: SQL value sets, ranges and renamed columns, a PromQL
offset window as relative time, a matcher below rate kept in the definition,
and a PromQL comparison above rate lifted into the selection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
…initions

`execution_timed_workload_dag` becomes `pub(crate)`: callers export a
planned workload through `PlanOutput::execution_timed_dag`, which also
carries its standalone scalar roots.

The coverage tests now check what each #573 §4.2.2 example's definition
keeps, not only its selection: the range example yields exactly one box and
a scan without the lifted predicate, the matcher below `rate` stays in the
definition's scan, and the definition keeps the `rate` under a lifted
`Filter`. The two missing rows are added: `rate(a[5m]) > rate(b[5m])` (a
`BinaryOp`, nothing selected) and `rate(m[5m]) > bool 0` (a `Project`,
kept in the definition).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
The #573 coverage tests no longer depend on which candidate the default
cost model selects: each takes the one KLL SummaryAgg among the search
space's candidates. The SQL tests also check that the definition's
SummaryAgg has no filter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
@zzylol
zzylol changed the base branch from stack/528-07b-promote-roots to main October 13, 2026 01:56
@zzylol
zzylol force-pushed the stack/528-07c-planner-followups branch from 37e7726 to ee9a3f5 Compare October 13, 2026 01:56
@zzylol
zzylol merged commit 05f282e into main Oct 13, 2026
3 of 4 checks passed
@zzylol zzylol mentioned this pull request Oct 13, 2026
2 tasks done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant