Skip to content

Commit a94286a

Browse files
authored
Merge pull request #239 from acgetchell/fix/benchmark-docs
fix: complete benchmark CI and improve documentation navigation
2 parents bd80cc0 + e7ca1fe commit a94286a

8 files changed

Lines changed: 1236 additions & 954 deletions

File tree

.github/workflows/benchmarks.yml

Lines changed: 24 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,9 @@ env:
5858
jobs:
5959
bench:
6060
runs-on: ubuntu-latest
61-
timeout-minutes: 30
61+
# The full exact suite has 260+ cases: Criterion's default 3 s warm-up
62+
# plus 5 s measurement already needs 35+ minutes, before build/analysis.
63+
timeout-minutes: 90
6264

6365
steps:
6466
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -109,8 +111,7 @@ jobs:
109111
repo: context.repo.repo,
110112
workflow_id: 'benchmarks.yml',
111113
branch: 'main',
112-
status: 'completed',
113-
conclusion: 'success',
114+
status: 'success',
114115
per_page: 5,
115116
});
116117
@@ -158,15 +159,16 @@ jobs:
158159
echo "::notice::Baseline found — comparing against main"
159160
echo "comparison_available=true" >> "$GITHUB_OUTPUT"
160161
# --baseline-lenient rather than --baseline: benches added on the
161-
# PR branch that don't yet exist in the main baseline get a
162-
# "no baseline data" notice instead of aborting the whole run.
162+
# PR branch that don't yet exist in the main baseline run without
163+
# a comparison instead of aborting the whole run.
164+
# Keep normal sampling; plots/HTML are unused by the summary.
163165
cargo bench --locked --features bench,exact --bench exact \
164-
-- --baseline-lenient main 2>&1 | tee bench-output.txt
166+
-- --noplot --baseline-lenient main 2>&1 | tee bench-output.txt
165167
else
166168
echo "::notice::No baseline found — running without comparison"
167169
echo "comparison_available=false" >> "$GITHUB_OUTPUT"
168170
cargo bench --locked --features bench,exact --bench exact \
169-
2>&1 | tee bench-output.txt
171+
-- --noplot 2>&1 | tee bench-output.txt
170172
fi
171173
172174
if grep -q "Performance has regressed" bench-output.txt; then
@@ -182,11 +184,11 @@ jobs:
182184
github.ref == 'refs/heads/main'
183185
run: >
184186
cargo bench --locked --features bench,exact --bench exact
185-
-- --save-baseline main
187+
-- --noplot --save-baseline main
186188
187189
- name: Run benchmarks (manual ref)
188190
if: github.event_name == 'workflow_dispatch' && github.ref != 'refs/heads/main'
189-
run: cargo bench --locked --features bench,exact --bench exact
191+
run: cargo bench --locked --features bench,exact --bench exact -- --noplot
190192

191193
- name: Upload baseline artifact
192194
if: >
@@ -208,12 +210,24 @@ jobs:
208210
comparison_available="${BENCH_COMPARISON_AVAILABLE:-}"
209211
regression="${BENCH_REGRESSION:-}"
210212
213+
# Lenient comparisons silently skip missing baselines. Require a
214+
# change report for every analyzed case before claiming coverage.
215+
if [ -f bench-output.txt ]; then
216+
analyzed="$(grep -c '^Benchmarking .*: Analyzing$' bench-output.txt || true)"
217+
compared="$(grep -c '^[[:space:]]*change:' bench-output.txt || true)"
218+
if [ "$analyzed" = "0" ] || [ "$analyzed" != "$compared" ]; then
219+
comparison_available=false
220+
fi
221+
else
222+
comparison_available=false
223+
fi
224+
211225
if [ "$comparison_available" != "true" ] || [ -z "$regression" ]; then
212226
{
213227
echo "### ❓ Benchmark Comparison Unavailable"
214228
echo ""
215229
echo "No usable comparison against the main baseline was produced."
216-
echo "The benchmark still ran, but no regression claim can be made."
230+
echo "Benchmarks may be incomplete or lack a baseline; no regression claim can be made."
217231
} >> "$GITHUB_STEP_SUMMARY"
218232
echo "::warning::Benchmark comparison unavailable"
219233
elif [ "$regression" = "true" ]; then

AGENTS.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -230,12 +230,32 @@ When user requests commit message generation:
230230

231231
- `src/lib.rs` includes `README.md` with `#![doc = include_str!("../README.md")]`, so README examples are the
232232
docs.rs landing page examples.
233+
- Keep the README quickstart and brief capability descriptions discoverable;
234+
put fuller worked API examples and caller contracts in the documentation-only
235+
`guide` module in `src/lib.rs`. Preserve useful detail in the linked guide when
236+
shortening README sections, including numerical limitations and error semantics.
237+
- Link API references and worked API guides to docs.rs. Keep repository-owned
238+
mathematical background, benchmark reports, roadmap, contributing, and release
239+
instructions on GitHub, using absolute URLs to the intended repository revision.
240+
- README links and Contents anchors must work both on GitHub and in generated
241+
rustdoc. Repository-relative file links can resolve incorrectly from rustdoc;
242+
verify destinations in both contexts and use intra-doc links within Rust docs
243+
where the referenced item is available under the selected features.
244+
- Choose docs.rs `latest` links for intentionally current guidance and explicit
245+
versions for release-specific contracts. docs.rs builds published crates, so
246+
merging changes does not publish new guide pages. Local rendering checks do
247+
not establish availability on the published site.
233248
- When changing Rust examples in `README.md`, mirror executable versions in the private `readme_doctests` module in
234249
`src/lib.rs`. Keep mirrors hidden/private so they do not duplicate the docs.rs landing page, but make them runnable
235250
by `cargo test --doc`.
236251
- README examples that require optional features may remain `rust,ignore` in README for default-feature doctest
237252
compatibility, but must have a `#[cfg(feature = "...")]` hidden doctest mirror in `src/lib.rs` and be verified with
238253
the matching feature set (for example, `cargo test --features exact --doc`).
254+
- Guide examples run directly as doctests; gate feature-dependent guides with
255+
the matching feature and remove obsolete private mirrors when moving examples
256+
out of README. Run `just doc-check` for changed guide docs and inspect generated
257+
pages and anchors for explicit links, which rustdoc does not validate. Validate
258+
changed executable examples with the default and matching feature doctest recipes.
239259
- When intentionally updating package versions or dependency snippets, keep README `la-stack` dependency examples in
240260
sync with the package `version` in `Cargo.toml`. Do not perform version bumps unless explicitly requested by the
241261
maintainer; see **Public-API stability** above.

0 commit comments

Comments
 (0)