Repository navigation
test: stream five slow tree-scan ratchets through source_corpus - #36648
Conversation
Code change approver recommendation: ApproveWhy: the decider estimates a 7% chance that this change introduces a defect, within the 12% limit for approving on its own, and every check passes. Test-only change that routes five tree-scan ratchets through source_corpus.parsed_candidates/iter_candidate_sources with name literals each AST pattern must spell. Filters only drop files that could never match (the matching is NFKC-aware and every literal is an identifier or attribute name). Every rewrite keeps its old population, exclusions, syntax-error handling and relative paths. The lenient ratchet drops module text only after the scan is done, and the later tests read only readers and the released tree fields. Its new tree_scan_ xdist group follows testing-conventions.md. Nothing regresses and no assertion is weakened. An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
1e7a776 to
603ea02
Compare
First Principles Review (Opus 5.5) — ⏭️ skippedRevision |
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: this is a test-only change, and the text filter can only skip files that could never match, so each ratchet still checks what it checked before. [DESIGN-REVIEWED] 2264d18 Description read[DESCRIPTION-READ] a6671f0f53c7588236677bfd6adebafa2f2d71d49551edbdfa8f5f656b27ef84 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Four ratchets parsed every module under src/ (one at collection time on every worker) although only files that spell the names they look for can match. They now go through source_corpus.parsed_candidates or iter_candidate_sources, which read one file at a time and parse only those candidates. The lenient-read ratchet already drops each tree; its cached scan now also drops the module text, and its tree-wide tests run as one xdist group so the scan runs once per run instead of once per worker. What each ratchet checks is unchanged; a planted violation still fails each one.
603ea02 to
2264d18
Compare
|
Rebased onto main after #36731 (Bolin Chen) landed the same per-tree release in |
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Approved. On this head the review read all five changed test files, and each new pre-filter is a name the AST check itself must spell, so a filter can only drop files that could never match. Each scan keeps its old behaviour: the _vendor/ and /tests/ exclusions, the dashboard root restriction, sorted file order, and SyntaxError handling (skip_syntax_errors=False where the old code raised). Old and new scans run over the head tree found the same sites in every case (35, 12, 4, 7, 13), and no assertion, count, known list or timeout changed. Done well: each ratchet was proven to still fail by planting a violation, and the failure named its site.
Optional:
- #36653 adds the same
xdist_groupline to test/test_lenient_read_rewrite_ratchet.py, and nothing else. This PR covers it, so only one of the two should land. - In
_src_scan, nothing tests themodule.text = Nonedrop. Addingor m.text is not Noneto theheldcheck intest_the_scan_keeps_no_parsed_treewould make the test fail if someone deletes that line. - Most of the lenient-row memory saving comes from #36731, which is already on main. This PR's own drop is 437 to 423 MiB, as your 17:59Z comment says.
Problem / Motivation
Goal: Five slow tree-scan ratchets run faster, and the largest one stops holding every parsed module in memory, while each still checks exactly what it checked before.
Some ratchets
rglob+ast.parseevery module undersrc/even though only files that spell the names they look for can match. One of them does this at collection time on every xdist worker. The lenient-read ratchet also kept every parsed module alive in its cached scan, which made it the biggest memory step on its shard.Why it matters
These files are among the slowest in the backend suite, so every CI run pays for them on every worker that collects or runs them.
test_lenient_read_rewrite_ratchet.pyis pinned to shard 7. That shard's runner already peaks at about 15.0 GB of 15.7 GB and is out-of-memory-killing node children there (#36261), so a 1 GB drop in one module's peak memory is real headroom.Not a goal
test/source_corpus.pyandtesting-conventions.mdrecord why retaining trees is the wrong fix. Only small extracted results are cached.test/conftest.py,source_corpus.py, anyscripts/gate, or security-held test files.What changed (motivation → approach → change)
Each scan goes through
source_corpus.parsed_candidates/iter_candidate_sources. These stream one file at a time and parse only the files whose NFKC-normalised text holds the literal the AST check needs. A file without that literal can never match, so the filter removes only non-matches.test_lenient_read_rewrite_ratchet.pyxdist_groupso the scan runs once per run, not once per workertest_persistence_writer_inventory.pytest_turn_trigger_source.pytest_slack_mirror_echo.py_run_chattest_decisions_model_route_app_actor.pyTimes are each file alone at
-n0on one host (16 cores, idle), including about 2 s of pytest start-up per file; the lenient row is measured against current main, which already carries #36731.Backwards compatibility
Compatible: test-only change. The ratchets scan the same population and assert the same things.
Removes nothing: no production code, pin, known-site list or assertion is removed. Only how the tests read the tree changes.
Tests
Proof: not provable -- test-only change
A violation was planted in the working tree for each ratchet, the ratchet was run, and the plant was restored from a byte copy. Every plant failed its ratchet and named the planted site:
_loadfeeds_writeinbump_planted_lenient.py::bumpstore.write_lesson_planted_writer.py::sneakypersist_token_record_asynccall withouttrigger=_planted_turn.pytest_the_source_scan_finds_every_known_system_producerfailsqueue_appendthat readsrequest_appwith no actor_planted_queue.pytest/test_source_corpus.py,-n 2: 174 passed.test/test_flake_pattern_ratchet.pyandtest/test_wait_discipline_ratchet.pyagainst the merge base: 122 passed.Manual verification
N/A — unit coverage sufficient: test-only change, proven by the planted violations above.
Related Issues
Part of #25195
Refs #36261
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)