Repository navigation
CI: measure and reduce core test work before extending its timeout #2182
Description
Problem
test-core-python runs nearly the full test suite in seven Python versions. The Python 3.14 cell also collects changed-line coverage. The current 8-minute job timeout leaves little margin on ordinary changes, and #2178 proposes raising it to 12 minutes. A timeout increase alone does not reduce the test work or release wait.
Evidence (11 Oct 2026)
- Recent master run 38102087533: Python 3.11 7m27s and Python 3.12 7m40s job durations, against the 8-minute limit. The latter's test step was 7m23s; dependency installation was ~10s.
- fix(gfql/index): enforce relationship uniqueness in indexed bindings (#2177) #2178 run 38113297389: Python 3.14 job 7m46s; the coverage test step was 7m24s.
- ci: replay GFQL routes only for affected tests #2181 run 38115093218: Python 3.14 coverage step was 4m35s. ci: replay GFQL routes only for affected tests #2181 optimizes route replay, a different job; this swing demonstrates that a single-hosted-run comparison cannot establish that fix(gfql/index): enforce relationship uniqueness in indexed bindings (#2177) #2178's new tests caused the slower run.
bin/test.shinvokes pytest over almost the entire repository on each matrix cell. fix(gfql/index): enforce relationship uniqueness in indexed bindings (#2177) #2178 adds one 164-line test module. We do not yet have per-module timing evidence to remove or optimize a specific test safely.
Requested work
Profile the core lane with stable per-module/per-test timing and compare repeated runs on matched source/runner conditions. Identify redundant or disproportionately costly cases, fixtures, collection or coverage work, then remove or optimize measured work while preserving the same meaningful version/behavior and changed-line coverage gates. Report total runner-minutes and critical-path wall time, not just a faster shard achieved by adding parallel jobs. Reassess the 8-minute deadline on an exact integration head after the improvement; keep enough measured variance margin.
Polars coverage capacity is separately tracked by #1797. Its issue body predates the current xdist runner; refresh that evidence before choosing additional Polars changes. #2181's route-replay efficiency is separate from these core and Polars jobs.
Acceptance
- A before/after repeatable per-module timing record identifies actual work removed or optimized.
- Core correctness, supported Python-version coverage and changed-line coverage behavior stay green.
- Measured runner-minutes and full workflow critical path improve; timeout value is justified by observed distribution.
Concrete first candidate (11 Oct 2026 hosted log audit)
In #2181 run 38115093218, Python 3.14 core job 114398757516 spent about 83.4 seconds on graphistry/tests/compute/gfql/cypher/test_differential_grammar_parity.py::test_full_repo_lalr_earley_differential, the largest single measured test on that worker. The dedicated cypher-frontend-differential-parity (py3.12) job 114398354515 also ran that exact test and passed (1 passed in 120.61s), finishing before the core job started. The core matrix has seven Python versions; repeated work is plausible, but sevenfold savings and workflow-wall gains have not yet been measured.
The full corpus scans every package .py, so removing core copies needs a retained gate on broad Python edits. Further audit found test-gfql-core already runs the full GFQL directory on Python 3.12 (with coverage) and 3.14 whenever generic core runs: exhaustive current-condition evaluation covered 2,048 path-filter/event states. #2183 therefore excludes the file only from the seven generic core cells and leaves the dedicated gate trigger untouched. Constructed grammar invariant tests remain in all core Python versions; the full-repository sweep no longer runs in generic core 3.8–3.11/3.13. Keep #2181's reviewed route-replay head unchanged and do not reduce timeouts before measured integration evidence.
#2183 hosted result (11 Oct 2026)
#2183 is now OPEN/non-draft at f86d2766d797f85704dc5f7bdab473badeae3e6e, with two quiet review waves and exact-head run 38123237023 complete: 79 successful checks and two expected path-gated skips. The baseline #2181 core logs directly attribute 19.86 runner-minutes to the seven repeated full-corpus sweep calls. The #2183 core matrix totaled 32.58 runner-minutes versus 40.85 (master schedule) and 41.23 (#2181 PR) in prior runs; the observed 20–21% group reduction is affected by runner variance. The workflow wall was 17m46s, but its path-triggered work differs from those baselines, so this is not a proven whole-workflow wall-time improvement.
The hosted core 3.8/3.14 logs show the sweep excluded and constructed grammar invariants retained; GFQL core 3.12/3.14 logs show the sweep passed. GFQL 3.12 per-file coverage audit reports cypher/parser.py 90.29% against 88.35% baseline. Changed-line coverage passed, but with 0 eligible changed package lines on this workflow-only PR; the retained sweep and per-file audit supply the relevant parser evidence. The version-scope trade-off and maintainer-only merge boundary are in the PR body. If #2181 lands first, rerun exact integrated CI before #2183 merge. The longer-term issue remains open for measured core work reduction beyond this one duplicate and for timeout-margin reassessment.
Activity