Skip to content

test(spec-builder): split the route tests' tail into behaviour parts - #20327

Merged
bolichen97 merged 1 commit into
mainfrom
refactor/linecap-t03-test-routes
Oct 14, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
refactor/linecap-t03-test-routes

Conversation

@bolichen97

@bolichen97 bolichen97 commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Bring the Spec Builder route tests (spec_builder/tests/test_routes.py, 17,296 lines) under the 2,000-line file cap by splitting them into behaviour-named parts, without changing any test; this PR is the first of two moves.

test_routes.py is the largest test file in the app suites: one module that pins the route contract, path containment, the index, the STOP sentinel, slot identity, deletes, duplicates, the decision ledger, aliases, the turn lock and stop barriers. A reader looking for "the tests of X" has to search 17k lines, and every new test lands in the same file. The line-cap campaign caps hand-written files at 2,000 lines.

Why it matters

A file this size cannot be reviewed or navigated as a unit, and it keeps growing because there is no other place for a new route test to go. Splitting it by the behaviour each test pins gives every topic one home, so the next test goes next to its siblings and the cap stays enforceable.

Not a goal

  • Changing any test's behaviour, name, parametrization or marker. Tests move byte-for-byte (one recorded exception below).
  • Fixing the pre-existing test weaknesses the move surfaces: two sleep barriers, the autouse guard on the shared monkeypatch, and test_redirect_state_covers_every_path_the_app_writes going vacuous for lazily-bound paths. They are reported to the campaign for one de-flake follow-up and marked in place with # nosemgrep (see Tests).
  • Finishing the split. A whole-file move is about 1.3 MB of diff, over the review lanes' cap, so the split ships as two PRs; the second moves the rest and leaves test_routes.py as the route contract, about 360 lines.
  • Touching production code under spec_builder/backend/.

What changed (motivation → approach → change)

The file grew one hardening round at a time, so its section comments record rounds, not behaviours: one section heading covers 7,282 lines of decision, alias, turn-lock and stop-barrier tests. Each test was therefore classified by the behaviour it exercises (read from its body), and the split is generated by a recipe script so it can be re-run on a newer main instead of hand-merged.

File What it holds after this PR
routes_helpers.py the harness tests in two or more parts share: state redirection, the autouse live-state guard, client builders, fakes and seeds
test_routes_aliases.py, test_routes_decision_answers.py, test_routes_decision_ledger.py, test_routes_decision_replay.py, test_routes_stop_barrier.py, test_routes_turn_lock.py, test_routes_artifacts.py complete topic parts
test_routes_duplicate.py, test_routes_create.py, test_routes_identity_pinning.py, test_routes_nudge_loops.py, test_routes_handoff.py, test_routes_delete.py the tail of their topic; the second PR adds the rest
test_routes.py BASE up to the "direct authority over the artifacts" section (8,147 lines); the second PR moves everything after its route-contract prefix

This PR moves everything from the "direct authority over the artifacts" section to the end of the file, so what test_routes.py keeps is one contiguous run of BASE: git's default diff and --minimal agree on it (24 added lines, the docstring and imports). The owner-gate tests at BASE's end go to test_routes_handoff.py (execute, handoff and stop) and the app-token test to test_routes_artifacts.py (the mutations' shared guards). test_routes_delete.py (121 lines) opens small on purpose: it holds the tail of its topic and the second PR fills it to about 980 lines.

How each part stays equivalent:

  • Fixtures: every part binds the autouse guard _never_touch_the_real_state in its own namespace (# noqa: F401, the meetings_helpers idiom); making the guard raise errors all 594 tests.
  • Importers: test_orchestration_composition_contract.py imports _BASE, _decision_spec, _make_client, _redirect_state, _slot_stub from test_routes.py; they still resolve there, to the same objects.
  • The one changed test body: test_state_guard_compares_the_real_dir_not_the_redirect reads routes_helpers._REAL_STATE_DIR. The guard's global assignment fills the helper module's name; a part's own import of it is the import-time None, and the verbatim body fails with the first-use capture did not run.
  • Section headers stay above the tests they introduce; the split gap 5 section is named for what each part holds.

Path-keyed rows: test/requires-real-symlinks.txt re-points the four moved node ids, and .github/file-line-cap-baseline.txt lowers this file's row to its new size. The Windows-gap name pin in test_conftest_windows_gaps_cov80.py reads every test_routes*.py file and asserts it found test_routes.py and at least one part. docs/system-specs/modules/app-kit-platform.md §19 says where a new route test goes. The spec_builder/tests/conftest.py docstring says "the route test files".

Backwards compatibility

Compatible: test-only. Every collected test keeps its name, parameters and markers (594 before and after); test_routes.py keeps every name another module imports from it.

Removes nothing: no production code, public function, route or data changes; the moved definitions are private test helpers, still importable from test_routes.py where another file imports them.

Tests

No test is added or rewritten. Evidence the move is faithful:

Collected multiset. pytest --collect-only -qq over BASE test_routes.py vs HEAD test_routes.py + test_routes_*.py: 594 = 594, identical (name, params) multiset.

Move audit (move_audit.py --origin .../test_routes.py, BASE = this PR's merge-base):

src/kiro_crew/apps/builtins/spec_builder/tests/test_routes.py: 17322 -> 8147 lines; moved 99.79% (python-ast)
   changed: Expr [Expr] 16 lines                       <- the module docstring
   changed: test_state_guard_compares_the_real_dir_not_the_redirect [FunctionDef] 16 lines   <- the recorded edit

Runs.

  • spec_builder/tests at -n 3 passes, and every part also passes run alone at -n0.
  • These repo guards pass: test_spec_builder_routes_coverage.py, test_source_pin_budget.py, test_auto_research_facade.py, test_lazy_data_home_paths.py, test_host_isolation_floor.py, test_mock_pid_gate_wiring.py, test_error_code_contract.py, test_security_posture.py, and test_flake_pattern_ratchet.py (-n0).
  • scripts/local-gate.py (run on an earlier head of this PR, same tests) selected 462 related files: 46,498 passed. 2 failed, both tree-scan tests timing out at 120 s on a loaded shared host: test_lazy_data_home_paths.py::...::test_no_module_level_path_constants and test_spawn_audit.py::test_every_spawn_is_routed_or_allowlisted. Both pass run alone (42 s and 45 s).
  • Static checks:
    • mypy src/kiro_crew is clean, and black, isort and flake8 are clean on the changed files.
    • CI's SAST config (semgrep 1.78, p/python p/typescript p/security-audit p/secrets p/owasp-top-ten semgrep/, --baseline-commit merge-base --error) reports 0 findings.
    • The comment-history, brand, harness-parity, memory-store, cwd-reads, loop-locks, testpaths and feature-map gates and docs_lint.py pass.
  • Coverage of each new file is 92-100%. An installed wheel under env -i imports routes_helpers and every part.

Mutation proofs.

  • Renaming one tracked Windows-gap test in test_routes_duplicate.py turns the re-pointed name pin red.
  • Replacing the guard's snapshot call in routes_helpers.py turns test_state_guard_watches_the_whole_directory red.

Moved findings.

  • Three pre-existing semgrep shapes are reported again in their new file by CI's diff-scoped scan. Each gets an own-line # nosemgrep: <rule> -- kept verbatim by the test_routes.py split above it:
    • kirocrew.test-autouse-shared-monkeypatch on the guard in routes_helpers.py;
    • kirocrew.test-sleep-as-barrier in test_routes_identity_pinning.py and test_routes_turn_lock.py.
  • Those two sleep barriers also carry an own-line # flake-ok: <reason> above them, the exemption the AUTOSDE tests-are-deterministic rule lists, so the review lanes read them as known and deferred rather than new.
  • The only credential-shaped literal that moves is AWS's documented example key AKIAIOSFODNN7EXAMPLE (test_routes_decision_answers.py, a redaction test).

Determinism: no test logic changed, so no new repeat proof applies; the flake-pattern ratchet passes.

Manual verification

N/A — test-only restructuring; the collected-test multiset, the AST move audit and the runs above are the verification.

Related Issues

Part of #20192

Follow-up: #20366 (pre-existing test weaknesses moved verbatim; de-flake there)

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@bolichen97
bolichen97 requested review from a team and KiroCrewCaptain as code owners October 14, 2026 14:03
@bolichen97
bolichen97 requested a review from mbriones98 October 14, 2026 14:03
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 14, 2026
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision c43e1849ec0fec8b332361451e53babf41755f9b ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Advisory — does not block merge.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of c43e1849ec0fec8b332361451e53babf41755f9b — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Check that the "tombstone race" test that testing-conventions.md attributes to test_routes.py still lives there; if it moved to a part, update the doc's file name.

[DESIGN-REVIEWED] c43e184

Description read

[DESCRIPTION-READ] 076bb90b4433a39af5fe90122758fd60a0053dfd1e6898b802ac9b9d44fac984

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision c43e1849ec0fec8b332361451e53babf41755f9b touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed c43e1849ec0fec8b332361451e53babf41755f9b — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] c43e184

Verdict parsed from the review's SHA-scoped output markers for commit c43e1849ec0fec8b332361451e53babf41755f9b.

False positive or not applicable? A repository writer can comment:
/ai-review override fable c43e1849ec0fec8b332361451e53babf41755f9b: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of c43e1849ec0fec8b332361451e53babf41755f9b and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] c43e184

False positive or not applicable? A repository writer can comment:
/ai-review override gpt c43e1849ec0fec8b332361451e53babf41755f9b: <one-sentence reason>

@bolichen97
bolichen97 force-pushed the refactor/linecap-t03-test-routes branch from 46a9855 to a145fb6 Compare October 14, 2026 14:32
@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=0b0c6b20996c Sleep barriers leave the moved race tests scheduler-dependent

fixed in a145fb6 with the rule's own exemption: both sites (test_routes_identity_pinning.py, test_routes_turn_lock.py) now carry an own-line # flake-ok: naming the barrier, above the existing # nosemgrep.
The bodies stay byte-identical on purpose: this PR moves tests verbatim (AST move audit 99.79%), and both sleeps are pre-existing at BASE test_routes.py:16406 and :16647.
Replacing them with parked-waiter signals is a test rewrite, recorded for the line-cap campaign's de-flake follow-up together with the other moved shapes (KC28 L37).
self-added: no
mechanism: own-line flake-ok marker on two moved sleep barriers

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • Punchline: confirm every part still triggers the _REAL_STATE_DIR first-use capture when run alone under -n0

rebutted (holds as-is): the capture runs in the autouse guard _never_touch_the_real_state, and every part binds that fixture in its own namespace, so it runs before every test of every part, alone or not.
Evidence: each test_routes_*.py passes run alone at -n0; making the guard raise errors all 594 collected tests; test_state_guard_compares_the_real_dir_not_the_redirect asserts the captured dir is set and is not the redirected one.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • Punchline: check that every part re-imports the autouse _never_touch_the_real_state guard

rebutted (holds today, and the gap is not a host write): every test_routes_*.py binds the guard in its helper import; making the guard raise at setup errors all 594 collected tests, so no part runs without it.
A future part that forgot the bind would still not reach the operator's state: _state_dir() and _decisions_path() resolve under config_dir(), which the rootdir conftest pins to a per-test KIROCREW_HOME; the guard is the second net, as routes_helpers.py's docstring says.
A test that enforces the bind on every part is new scope for this move-only PR; it is noted for the campaign follow-up.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 14, 2026

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes for one tracking gap. The move itself looks faithful.

The split matches what it claims. Every part binds the autouse live-state guard. test_routes.py keeps the names test_orchestration_composition_contract.py imports. The Windows-gap pin now globs every route file, and the spec_builder conftest skips by test name, so moved tests keep their Windows skips. All current-head checks are green, and the GPT, Opus, Design and First Principles lanes have no unanswered items.

Blocker: the PR promises a de-flake follow-up but links no GitHub issue for it. Not a goal and the in-code markers say the two sleep barriers, the autouse guard on the shared monkeypatch, and the vacuous test_redirect_state_covers_every_path_the_app_writes are 'tracked for a de-flake pass':

Please open one GitHub issue per remaining item (de-flake of the three shapes; the guard-bind check, unless you drop it as unneeded) and link them in the PR description. No code change is needed.

Reviewed head: a145fb6

— crew-pr-reviewer (r100)

@bolichen97

Copy link
Copy Markdown
Contributor Author

Thanks for the review of a145fb6. Both tracking items:

  1. De-flake of the moved shapes: filed as test(spec_builder): route tests wait on fixed sleeps, and the state-redirect check misses lazily bound paths #20366, now linked under ## Related Issues ("Follow-up: test(spec_builder): route tests wait on fixed sleeps, and the state-redirect check misses lazily bound paths #20366"). It covers the two sleep barriers (test_routes_identity_pinning.py, test_routes_turn_lock.py), the autouse guard on the shared monkeypatch (routes_helpers.py), and test_redirect_state_covers_every_path_the_app_writes going vacuous for lazily bound paths. All four are pre-existing at BASE and move verbatim here; the rewrite happens there. No code change in this PR.

  2. The guard-bind check: dropped as unneeded rather than deferred.

    • Every part binds _never_touch_the_real_state today. The proof: making the guard raise at setup errors all 594 collected tests, so no test of any part runs without it.
    • A future part that forgot the bind still could not reach the operator's state. _state_dir() and _decisions_path() resolve under config_dir(), and the rootdir conftest pins that to a per-test KIROCREW_HOME, so a missed bind writes only into that test's own home. The guard is the second net.
    • The earlier Design disposition said "noted for the campaign follow-up". This supersedes it: no issue is filed for the bind check.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: passed Eligible automated validation passed for the current revision labels Oct 14, 2026
test_routes.py is 17,296 lines, eight times the 2,000-line cap the
line-cap campaign enforces. This is the first of two moves. From the
"direct authority over the artifacts" section to the end of the file,
every test that belongs to a topic part moves, unchanged, into a sibling
test_routes_<topic>.py named for the behaviour it pins; the app-token and
owner-gate tests there are route-contract tests and stay. The harness
shared by more than one part (state redirection, the autouse live-state
guard, the client builders, fakes and seeds) moves into
routes_helpers.py. test_routes.py keeps the route contract and the tests
the second move takes.

The collected test multiset is unchanged (594 tests). One test body
changes: the state-guard self-test reads routes_helpers._REAL_STATE_DIR,
because the guard's global assignment now fills that module's name.
test_routes.py still binds the five names the composition contract
imports from it. The Windows-gap name pin reads every test_routes*.py
file, and the four symlink-list node ids that moved are rewritten.

Part of #20192
@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Oct 14, 2026
@bolichen97
bolichen97 force-pushed the refactor/linecap-t03-test-routes branch from a145fb6 to c43e184 Compare October 14, 2026 18:32
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 14, 2026
@bolichen97

Copy link
Copy Markdown
Contributor Author

Scheduled main refresh: rebased onto 3899beb (was c50003c). The PR had gone DIRTY on #20346, which edited tests this PR moves; its hunks are ported unchanged into test_routes_duplicate.py, test_routes_identity_pinning.py and test_routes_nudge_loops.py by re-applying the split recipe, and this file's line-cap baseline row is lowered to 8,147. CI re-running.

@github-actions github-actions Bot removed the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Oct 14, 2026
@bolichen97

Copy link
Copy Markdown
Contributor Author
  • Punchline: check that the "tombstone race" test testing-conventions.md attributes to test_routes.py still lives there

rebutted (accurate at this head): test_concurrent_tombstone_writes_do_not_lose_deletions is still in spec_builder/tests/test_routes.py; this PR moves only BASE's tail, from the "direct authority over the artifacts" section on.
The second PR of the series moves it to test_routes_delete.py and changes the citation in docs/system-specs/common/testing-conventions.md in that same commit.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 14, 2026

@mbriones98 mbriones98 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checks green and readiness passed at c43e184; no behavior change.

@bolichen97
bolichen97 enabled auto-merge (squash) October 14, 2026 22:59
@bolichen97
bolichen97 merged commit b87adec into main Oct 14, 2026
103 checks passed
@bolichen97
bolichen97 deleted the refactor/linecap-t03-test-routes branch October 14, 2026 23:04
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants