feat(asap-tools): run sketch-bench workload tables end to end with the paced feeder

[zzylol]

Stacked on #828 (--planner saved, controller.prebuilt_output_dir); base is its branch. This is #824 B4, revised: the data comes from the paced feeder instead of fake_exporter.

Summary

Runs a sketch-bench #777 workload table end to end with experiment_run_e2e.py. The paced feeder pushes the table's data in real time, and queries repeat at each RQE's interval T at evaluation times aligned to T.

• Feeder service (experiment_utils/services/paced_feeder.py), configured by exporters.exporter_list.paced_feeder: {workload_table, metrics, seed, duration_secs, samples_per_sec?, extra_flags?}:
  • One feeder process per (sink, metric), on the worker nodes (the coordinator in local mode). All processes get the same seed, table and --start-ms, so every system receives identical data.
  • sketchdb mode pushes to ASAPQuery's ingest, and to Prometheus too with query_prometheus_too; other modes push to Prometheus. In sketchdb mode Prometheus no longer forwards the fed metrics to ASAPQuery, so they don't arrive twice.
  • It starts once its sinks are ready, and fails the run if a feeder dies at startup (wrong URL, etc.).
  • Startup checks: the metric is in the table and is a gauge/counter, streaming.parallelism == 1, and duration_secs outlasts the queries.
  • setup_dependencies.sh builds the feeder.
• Prometheus: when the feeder is configured, it starts with --web.enable-remote-write-receiver and storage.tsdb.out_of_order_time_window: 5s. The feeder sends each one-second tick concurrently (fix(asap-tools): send paced feeder ticks concurrently, ordered per series, in (t, t+1s] #840), so a series' samples can arrive out of order within a tick, which Prometheus otherwise rejects. Bare metal and container both read the generated prometheus.yml.
• Query-time alignment: client_options.align_to_interval evaluates repetition k at floor((start − O)/T)·T + k·T and sends it O = query_time_offset seconds later, so all systems answer the same windows.
• Config generator (sketch_bench_e2e/sketch_bench_e2e_configs.py): from a workload table (+ optional prebuilt controller output dir), it writes an experiment_type YAML. Query groups come from e2e.queries grouped by interval_secs (the same grouping as --planner saved), with metrics from e2e.metrics and the feeder block.
• ASAPQuery's allowed lateness in these runs is the engine default, 5 s (the framework never overrides it), so the feeder's within-tick reordering is safe.

Not covered yet

• Multigroup (records) metrics (feeder support in feat(asap-tools): generate records metrics in the paced feeder #843) are fed to ASAPQuery only, as <table>.<field> for its SQL frontend. The startup check refuses them when any mode also sends to Prometheus (baseline, or query_prometheus_too). The generator feeds them only with --sql-queries, and then writes a sketchdb-only config; the Prometheus baseline for the classic queries of an all table is a separate run.
• SQL query groups: the query client uses one protocol per server, so multigroup's SQL queries can't be sent to ASAPQuery's SQL endpoint from this runner yet. The generator includes them only with --sql-queries FILE.
• No cluster run yet. The end-to-end check (ASAPQuery vs Prometheus vs ClickHouse on the same data) comes with the rest of the stack.

Testing

• 29 new tests: tests/test_paced_feeder.py (feeder command lines, sinks per mode, startup checks, the Prometheus receiver flag and out-of-order window via the real config generator), tests/test_sketch_bench_e2e_configs.py (config generation from a real classic table snippet) and queriers/prometheus-client/tests/test_query_time.py (alignment math). The experiments suite passes (86 tests) under pytest and unittest. A CI step now runs the prometheus-client tests. black/flake8 are clean.
• Dry run: the generated YAML for sketch-bench's classic shared1 table loads through Hydra and passes the runner's startup checks; controller, client and Prometheus configs are generated. In dry-run mode the client, started at 12:00:35 with T = 60 s and a 10 s offset, evaluated at 12:00:00, 12:01:00 and 12:02:00.

🤖 Generated with Claude Code

https://claude.ai/code/session_01AnJ7LocWxguh3q4QWkCYae