Repository navigation
ci(benchmark): emit reST footnotes and the engines page in the report, and size the timeout from the first run - #2142
Merged
Conversation
…, and size the timeout from the first run - report.py: rows and columns that need a note carry an auto-symbol footnote reference, [*]_, with the reason as the body directly after its table; overlapping rows share one labelled [#overlap]_ note. The literal markers and hand-written legends are gone from the RST, and the comment that justified them is corrected: docutils numbers footnotes page-wide, so nothing on the page collides. A gap no environment explained still gets a body. - report.py: the per-version lead names the script that ran (--runner) and, on Actions, the workflow (--workflow); native.sh passes both and run.sh passes its name through BENCH_RUNNER to entrypoint.sh. Every docs path now says docs/source/engines.rst. - README.md: points at engines.rst and describes the footnote output. - benchmark.yml: timeout-minutes 300 -> 90, from run 40062653380's 38-minute job on ubuntu-26.04-arm. Run 40062653380's documents now render the page's Test Results sections word for word. Harness and workflow tests pass.
Owner
Author
|
GOOD TO GO at
|
Contributor
|
Coverage: 91.55% (unit tier, Python 3.15,
Per-file detail: the |
Owner
Author
|
Ready to merge at
|
4 of 5 tasks
5 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
make pylint,make mypy,make isort)make testpasses, and a test case covers the changeWhat is the purpose of your pull request?
ci— workflows or build toolingDescription of your pull request and other information
Follow-up to #2128 and the new
docs/source/engines.rstpage (f9de043d6,d7aa41535), so a benchmark report pastes onto the page without hand edits.examples/benchmark/report.pyfootnotes: it emits reST footnotes instead of literal†/‡/¶markers and hand-written legends. Each noted row or column gets[*]_, overlapping rows share a labelled[#overlap]_, and the bodies follow each table in reference order. A gap with no recorded reason now gets an explicit "no reason recorded" note instead of a dangling marker.report.pyattribution: the "Measured with" line names the script that actually ran. The new--runner/--workflowoptions are fed bynative.shand, throughentrypoint.sh, byrun.sh. Everyindex.rstreference becomesengines.rst, in the script and inexamples/benchmark/README.md.benchmark.yml:timeout-minutesdrops from 300 to 90. The first full run, 40062653380 onubuntu-26.04-armwith every engine, took about 38 minutes.TestFootnotesinexamples/benchmark/test_harness.pyparses the output with docutils, so each reference must reach its own row's note. Againstmain, 20 tests fail. A new test pins the timeout.engines.rst, changes no words. Only two tables' first-column padding differs.