Repository navigation
test: stop holding every parsed src module in the argv-floor derivations - #38935
Conversation
First Principles Review (Opus 5.5) — ⏭️ skippedRevision |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks. This is a test-only change that swaps a cached AST for cached import tuples, and every check the derivations make stays the same. [DESIGN-REVIEWED] 61463bc Description read[DESCRIPTION-READ] 3c018e9bbb7ed0b8812aafbaad193152bb9ad9b2c8200ef2f9aeb9611be41df7 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
The two mint-surface derivations shared a module-scoped view of the product tree that kept the AST of every candidate module until the module ended, about 240 MiB of live memory per worker. Each tree is now parsed for the caller that asks and then dropped; only the import lines the import-graph walk reads again are kept, as plain tuples. Every check and derivation is unchanged.
550ce12 to
61463bc
Compare
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Approved. Review checks on head 61463bc:
- The rewritten
import_fromgives the same output in the same order. It walks the same sorted paths, uses the same byte filter and builds the same(module, name, bound)tuples. - Modules that do not parse are still skipped.
- The
producers_innot-None assertion still holds. - No assertion, seed or table changed, and nothing outside the test helper changed.
Done well: only the small per-module import tuples stay cached and the full ASTs are dropped. The planted re-export run through the new path shows the checks still fire.
Optional:
- The body cites
tools/cost-baseline/memory.md, but that file is not in the repository. The before/after RSS table already supports the goal, so you could drop that reference. _imports_ofcould store[]for an unparseable module. That would remove the| Noneannotations and theor ()at its one call site.
Code change approver recommendation: A person should decideWhy: the decider estimates a 4% chance that this change introduces a defect, within the 12% limit for approving on its own, but it is held at tier pending: waiting for the review lanes to finish. Test-only memory optimization of _ProductTree. tree() now parses for each caller instead of caching ASTs. import_from() keeps the same text filter and caches only per-module ImportFrom tuples, which don't depend on the target. The tuples, their walk order, the skipping of modules that don't parse, and the order across modules are all unchanged, so both derivations compute the same results. producers_in re-parses its module, which is deterministic, so its assert still holds. No other users of the helper, no src/ changes, and CI is green. Tier: pending: waiting for the review lanes to finish. An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
Problem / Motivation
Goal: Stop the argv-floor derivation tests from holding every parsed
src/module in memory, keeping every check they make.test_argv_floor_inline_and_brace_scope.pyshares a module-scoped_ProductTreebetween its two mint-surface derivations. It kept the AST of every candidate module it parsed until the module ended, so the two tests grew the worker's memory by about 240 MiB (tools/cost-baseline/memory.mdrow 4 measured 282 MiB peak growth).Why it matters
A worker's peak memory is what the xdist budget has to reserve. This file alone lifted a worker from ~200 MiB to ~450 MiB.
Not a goal
verify_only/not_readerstables.src/(the denied-command floor) ortest/conftest.py.What changed (motivation → approach → change)
_ProductTree.tree()parsed a module once and kept its AST to module endimport_from()walked each candidate's kept ASTfrom X import ylines, kept as plain tuples, parsed onceThe text filter that picks candidate modules is unchanged, and so is every AST walk the two tests do. The import-graph fixed point asks
import_frommany times, so its lines are the one thing kept;producers_inre-parses the few modules it needs.Backwards compatibility
Compatible: test-only change.
Removes nothing: no test, case, assertion or table entry is dropped.
Tests
Proof: not provable -- test-only change
Same host, Linux, the file alone at
-n 0, no coverage, main to branch (two runs each):VmHWM)Under
--cov=kiro_crew: peak 457-458 MiB to 235 MiB, wall 42.4-44.4 s to 42.2-43.5 s.Planted bugs, each run on this branch and on main (
cda68bf2b):inline_payload._MINT_VERB_REdropssecretinline_payload._MINT_VERB_REdropstokengenerate_tokenasmake_passtest_the_mint_surface_covers_every_token_producer_in_the_treefails, namingfrom kiro_crew.planted_reexport import make_passThe third plant goes through the rewritten
import_fromand the re-export fixed point.Coverage, the file at
-n 0 -p no:randomly: TOTAL 870570 statements / 776934 missed on both trees, every per-file row equal.Determinism: 3 shuffled orders of the 243 items and 5 repeats at
-n 0all pass. Flake-pattern ratchet, comment-history, brand and black gates pass. #36096 adds oneverify_onlyrow in this file; it touches no line this PR changes.Manual verification
N/A — test-only change; the runs above are the evidence.
Related Issues
N/A — test-memory work from
tools/cost-baseline/memory.md.Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)