Overview
The smoke runner's per-entry timings exist only as [PASS] <name> — <n>s lines in job logs, recovered by hand-scraping every time someone asks what is slow. Since the smoke-runner delegation (#260–#263) all ten workspace runners are thin shims over autohands/run_python.py, so recording per-script timings routinely is one PyAutoHands change, not ten repo sweeps. Phase 2 of the test-performance board arc (PyAutoLabs/PyAutoHeart#163); answers item 4 of the Mind's draft/research/ci/smoke_timing_and_profiling.md with "yes". Mind prompt: active/smoke_timings_dataset.md.
Plan
- The report machinery (
result_collector.RunReport, already mandatory in the PR gate via --report-dir) additionally emits a consolidated smoke_timings.json: one entry per executed script/notebook — {entry, kind, status, seconds, cap_s, exit_code} — plus run metadata (project, env profile, python version).
- When
$GITHUB_STEP_SUMMARY is set, append a compact per-entry timing table (slowest first) so every gate run's timings are one click away with no artifact download.
- PyAutoHeart's reusable
smoke-tests.yml uploads the report dir as a run artifact (if: always(), if-no-files-found: ignore) — the dataset persists the artifact retention window for every gate run across all ten repos (separate small PR on PyAutoHeart).
- Timing comes from the runner's own measurement, never re-derived; TIMEOUT entries record the cap they hit; skips are never silently timed as 0.
- Existing report consumers (
run_all.py, Heart test_run/script_timing) unaffected.
Detailed implementation plan
Affected Repositories
- PyAutoHands (primary)
- PyAutoHeart (one artifact-upload step in the reusable workflow, separate PR)
Branches
- PyAutoHands:
claude/test-performance-dashboard-y3fdy7 (from main @ e510de1)
- PyAutoHeart:
claude/smoke-timings-artifact-y3fdy7 (the phase-0 fix holds the main task branch there)
Implementation Steps
autohands/result_collector.py — RunReport gains per-entry timing capture (reusing the durations it already records for the per-script JSONs) and a write_timings() leg emitting smoke_timings.json alongside the existing report files; step-summary table writer guarded on GITHUB_STEP_SUMMARY.
autohands/run_python.py / run_notebook.py — no interface change; the report path emits the new file whenever --report-dir is passed.
PyAutoHeart/.github/workflows/smoke-tests.yml — upload-artifact step for the report dir.
- Tests: entry shape incl. TIMEOUT carrying
cap_s; skips absent/marked; step-summary table golden test; existing report consumers untouched (full suite green).
Original Prompt
Click to expand starting prompt
See PyAutoMind/draft/feature/pyautohands/smoke_timings_dataset.md (moved to active/ on issue) — full task text and acceptance criteria there.
Overview
The smoke runner's per-entry timings exist only as
[PASS] <name> — <n>slines in job logs, recovered by hand-scraping every time someone asks what is slow. Since the smoke-runner delegation (#260–#263) all ten workspace runners are thin shims overautohands/run_python.py, so recording per-script timings routinely is one PyAutoHands change, not ten repo sweeps. Phase 2 of the test-performance board arc (PyAutoLabs/PyAutoHeart#163); answers item 4 of the Mind'sdraft/research/ci/smoke_timing_and_profiling.mdwith "yes". Mind prompt:active/smoke_timings_dataset.md.Plan
result_collector.RunReport, already mandatory in the PR gate via--report-dir) additionally emits a consolidatedsmoke_timings.json: one entry per executed script/notebook —{entry, kind, status, seconds, cap_s, exit_code}— plus run metadata (project, env profile, python version).$GITHUB_STEP_SUMMARYis set, append a compact per-entry timing table (slowest first) so every gate run's timings are one click away with no artifact download.smoke-tests.ymluploads the report dir as a run artifact (if: always(),if-no-files-found: ignore) — the dataset persists the artifact retention window for every gate run across all ten repos (separate small PR on PyAutoHeart).run_all.py, Hearttest_run/script_timing) unaffected.Detailed implementation plan
Affected Repositories
Branches
claude/test-performance-dashboard-y3fdy7(frommain@e510de1)claude/smoke-timings-artifact-y3fdy7(the phase-0 fix holds the main task branch there)Implementation Steps
autohands/result_collector.py— RunReport gains per-entry timing capture (reusing the durations it already records for the per-script JSONs) and awrite_timings()leg emittingsmoke_timings.jsonalongside the existing report files; step-summary table writer guarded onGITHUB_STEP_SUMMARY.autohands/run_python.py/run_notebook.py— no interface change; the report path emits the new file whenever--report-diris passed.PyAutoHeart/.github/workflows/smoke-tests.yml— upload-artifact step for the report dir.cap_s; skips absent/marked; step-summary table golden test; existing report consumers untouched (full suite green).Original Prompt
Click to expand starting prompt
See
PyAutoMind/draft/feature/pyautohands/smoke_timings_dataset.md(moved toactive/on issue) — full task text and acceptance criteria there.