Repository navigation
test(spec-builder): split the route tests' tail into behaviour parts - #20327
Conversation
First Principles Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Check that the "tombstone race" test that [DESIGN-REVIEWED] c43e184 Description read[DESCRIPTION-READ] 076bb90b4433a39af5fe90122758fd60a0053dfd1e6898b802ac9b9d44fac984 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
46a9855 to
a145fb6
Compare
|
|
|
iamwhatever
left a comment
There was a problem hiding this comment.
Requesting changes for one tracking gap. The move itself looks faithful.
The split matches what it claims. Every part binds the autouse live-state guard. test_routes.py keeps the names test_orchestration_composition_contract.py imports. The Windows-gap pin now globs every route file, and the spec_builder conftest skips by test name, so moved tests keep their Windows skips. All current-head checks are green, and the GPT, Opus, Design and First Principles lanes have no unanswered items.
Blocker: the PR promises a de-flake follow-up but links no GitHub issue for it. Not a goal and the in-code markers say the two sleep barriers, the autouse guard on the shared monkeypatch, and the vacuous test_redirect_state_covers_every_path_the_app_writes are 'tracked for a de-flake pass':
-
#20192 covers finishing the split but not these items. The Design disposition also defers a check that every part binds the guard.
Please open one GitHub issue per remaining item (de-flake of the three shapes; the guard-bind check, unless you drop it as unneeded) and link them in the PR description. No code change is needed.
Reviewed head: a145fb6
— crew-pr-reviewer (r100)
|
Thanks for the review of a145fb6. Both tracking items:
|
test_routes.py is 17,296 lines, eight times the 2,000-line cap the line-cap campaign enforces. This is the first of two moves. From the "direct authority over the artifacts" section to the end of the file, every test that belongs to a topic part moves, unchanged, into a sibling test_routes_<topic>.py named for the behaviour it pins; the app-token and owner-gate tests there are route-contract tests and stay. The harness shared by more than one part (state redirection, the autouse live-state guard, the client builders, fakes and seeds) moves into routes_helpers.py. test_routes.py keeps the route contract and the tests the second move takes. The collected test multiset is unchanged (594 tests). One test body changes: the state-guard self-test reads routes_helpers._REAL_STATE_DIR, because the guard's global assignment now fills that module's name. test_routes.py still binds the five names the composition contract imports from it. The Windows-gap name pin reads every test_routes*.py file, and the four symlink-list node ids that moved are rewritten. Part of #20192
a145fb6 to
c43e184
Compare
|
Scheduled main refresh: rebased onto 3899beb (was c50003c). The PR had gone DIRTY on #20346, which edited tests this PR moves; its hunks are ported unchanged into test_routes_duplicate.py, test_routes_identity_pinning.py and test_routes_nudge_loops.py by re-applying the split recipe, and this file's line-cap baseline row is lowered to 8,147. CI re-running. |
|
mbriones98
left a comment
There was a problem hiding this comment.
Checks green and readiness passed at c43e184; no behavior change.
Problem / Motivation
Goal: Bring the Spec Builder route tests (
spec_builder/tests/test_routes.py, 17,296 lines) under the 2,000-line file cap by splitting them into behaviour-named parts, without changing any test; this PR is the first of two moves.test_routes.pyis the largest test file in the app suites: one module that pins the route contract, path containment, the index, the STOP sentinel, slot identity, deletes, duplicates, the decision ledger, aliases, the turn lock and stop barriers. A reader looking for "the tests of X" has to search 17k lines, and every new test lands in the same file. The line-cap campaign caps hand-written files at 2,000 lines.Why it matters
A file this size cannot be reviewed or navigated as a unit, and it keeps growing because there is no other place for a new route test to go. Splitting it by the behaviour each test pins gives every topic one home, so the next test goes next to its siblings and the cap stays enforceable.
Not a goal
monkeypatch, andtest_redirect_state_covers_every_path_the_app_writesgoing vacuous for lazily-bound paths. They are reported to the campaign for one de-flake follow-up and marked in place with# nosemgrep(see Tests).test_routes.pyas the route contract, about 360 lines.spec_builder/backend/.What changed (motivation → approach → change)
The file grew one hardening round at a time, so its section comments record rounds, not behaviours: one section heading covers 7,282 lines of decision, alias, turn-lock and stop-barrier tests. Each test was therefore classified by the behaviour it exercises (read from its body), and the split is generated by a recipe script so it can be re-run on a newer main instead of hand-merged.
routes_helpers.pytest_routes_aliases.py,test_routes_decision_answers.py,test_routes_decision_ledger.py,test_routes_decision_replay.py,test_routes_stop_barrier.py,test_routes_turn_lock.py,test_routes_artifacts.pytest_routes_duplicate.py,test_routes_create.py,test_routes_identity_pinning.py,test_routes_nudge_loops.py,test_routes_handoff.py,test_routes_delete.pytest_routes.pyThis PR moves everything from the "direct authority over the artifacts" section to the end of the file, so what
test_routes.pykeeps is one contiguous run of BASE: git's default diff and--minimalagree on it (24 added lines, the docstring and imports). The owner-gate tests at BASE's end go totest_routes_handoff.py(execute, handoff and stop) and the app-token test totest_routes_artifacts.py(the mutations' shared guards).test_routes_delete.py(121 lines) opens small on purpose: it holds the tail of its topic and the second PR fills it to about 980 lines.How each part stays equivalent:
_never_touch_the_real_statein its own namespace (# noqa: F401, themeetings_helpersidiom); making the guard raise errors all 594 tests.test_orchestration_composition_contract.pyimports_BASE,_decision_spec,_make_client,_redirect_state,_slot_stubfromtest_routes.py; they still resolve there, to the same objects.test_state_guard_compares_the_real_dir_not_the_redirectreadsroutes_helpers._REAL_STATE_DIR. The guard'sglobalassignment fills the helper module's name; a part's own import of it is the import-timeNone, and the verbatim body fails withthe first-use capture did not run.gap 5section is named for what each part holds.Path-keyed rows:
test/requires-real-symlinks.txtre-points the four moved node ids, and.github/file-line-cap-baseline.txtlowers this file's row to its new size. The Windows-gap name pin intest_conftest_windows_gaps_cov80.pyreads everytest_routes*.pyfile and asserts it foundtest_routes.pyand at least one part.docs/system-specs/modules/app-kit-platform.md§19 says where a new route test goes. Thespec_builder/tests/conftest.pydocstring says "the route test files".Backwards compatibility
Compatible: test-only. Every collected test keeps its name, parameters and markers (594 before and after);
test_routes.pykeeps every name another module imports from it.Removes nothing: no production code, public function, route or data changes; the moved definitions are private test helpers, still importable from
test_routes.pywhere another file imports them.Tests
No test is added or rewritten. Evidence the move is faithful:
Collected multiset.
pytest --collect-only -qqover BASEtest_routes.pyvs HEADtest_routes.py+test_routes_*.py: 594 = 594, identical(name, params)multiset.Move audit (
move_audit.py --origin .../test_routes.py, BASE = this PR's merge-base):Runs.
spec_builder/testsat-n 3passes, and every part also passes run alone at-n0.test_spec_builder_routes_coverage.py,test_source_pin_budget.py,test_auto_research_facade.py,test_lazy_data_home_paths.py,test_host_isolation_floor.py,test_mock_pid_gate_wiring.py,test_error_code_contract.py,test_security_posture.py, andtest_flake_pattern_ratchet.py(-n0).scripts/local-gate.py(run on an earlier head of this PR, same tests) selected 462 related files: 46,498 passed. 2 failed, both tree-scan tests timing out at 120 s on a loaded shared host:test_lazy_data_home_paths.py::...::test_no_module_level_path_constantsandtest_spawn_audit.py::test_every_spawn_is_routed_or_allowlisted. Both pass run alone (42 s and 45 s).mypy src/kiro_crewis clean, and black, isort and flake8 are clean on the changed files.semgrep 1.78,p/python p/typescript p/security-audit p/secrets p/owasp-top-ten semgrep/,--baseline-commitmerge-base--error) reports 0 findings.docs_lint.pypass.env -iimportsroutes_helpersand every part.Mutation proofs.
test_routes_duplicate.pyturns the re-pointed name pin red.routes_helpers.pyturnstest_state_guard_watches_the_whole_directoryred.Moved findings.
# nosemgrep: <rule> -- kept verbatim by the test_routes.py splitabove it:kirocrew.test-autouse-shared-monkeypatchon the guard inroutes_helpers.py;kirocrew.test-sleep-as-barrierintest_routes_identity_pinning.pyandtest_routes_turn_lock.py.# flake-ok: <reason>above them, the exemption the AUTOSDEtests-are-deterministicrule lists, so the review lanes read them as known and deferred rather than new.AKIAIOSFODNN7EXAMPLE(test_routes_decision_answers.py, a redaction test).Determinism: no test logic changed, so no new repeat proof applies; the flake-pattern ratchet passes.
Manual verification
N/A — test-only restructuring; the collected-test multiset, the AST move audit and the runs above are the verification.
Related Issues
Part of #20192
Follow-up: #20366 (pre-existing test weaknesses moved verbatim; de-flake there)
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)