Skip to content

test: split the rest of test_cli.py into topic files (2 of 2) - #22152

Merged
iamwhatever merged 1 commit into
mainfrom
refactor/linecap-t59-test-cli-p2
Oct 20, 2026
Merged

iamwhatever merged 1 commit into
mainfrom
refactor/linecap-t59-test-cli-p2

Conversation

@bolichen97

@bolichen97 bolichen97 commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Bring test/test_cli.py from about 9,500 lines to about 620, under the 2,000-line cap, by moving its tests into topic files, without changing what any test checks.

test/test_cli.py is the CLI's largest test file and is far over the 2,000-line cap scripts/check_file_line_cap.py enforces (it is listed in .github/file-line-cap-baseline.txt). The first PR of this series moved its kirocrew chat permission tests; this second one moves the rest of the commands' tests into one file per command or concern, and deletes the file's line-cap row.

Why it matters

The maintainers' rule is that no hand-written file may exceed 2,000 lines. This file changes most weeks, and every new test makes it harder to navigate and review. One file per command (cron, setup, stop, restart, token, doctor sections) makes each topic findable by name and gives each file room to grow.

Not a goal

  • Changing, renaming, fixing, de-flaking or re-parametrizing any test. Every test body moves byte for byte, apart from the few listed lines the gates require.
  • Changing production code. This PR touches only tests, their helper module, two baseline rows and the citations that name where these tests live.
  • Fixing the pre-existing timing shapes and semgrep findings that move with their tests (they are marked, not rewritten).

What changed (motivation → approach → change)

The file splits along its own test classes, grouped by the command or concern under test. A recipe tool did the move, reading the file at BASE, so it can be re-run after a rebase instead of hand-merged. The few classes whose text the added-line gates cannot judge stay in test_cli.py.

File (lines) What it holds
test/test_cli.py (9,533 → 621) the doctor's full report (TestDoctor), the update failure paths, the banner spelling, the config set value grammar
test_cli_cron.py (763), test_cli_setup.py (737) kirocrew cron add, update, remove; the setup wizard's prompts, the Slack manifest, the prerequisite check, the files under config_dir(), channel gating
test_cli_main_entry.py (1,021) cli.main()'s entry checks, builtin and --seed dispatch, the gateway lock-refusal exit, the asyncio child watcher
test_cli_gateway_client.py (880), test_cli_token.py (1,088) the client port lookup, the local-secret identity gate, and logout, status, spawn, artifact; kirocrew token, the post-restart token URL, the owner-verified secret transport
test_cli_stop.py (914) kirocrew stop and the gateway-process classifiers
test_cli_restart.py (1,250), test_cli_restart_readiness.py (765) kirocrew restart's dispatch, stop and spawn; what restart waits for after the spawn and reports
test_cli_mcp_doctor.py (803), test_cli_stt_memory_doctor.py (805) the doctor's MCP section; its Speech-to-Text and Vector Memory sections and the stale project dir
test/cli_helpers.py (210 → 319) the doctor tests' helpers, shared by test_cli.py and the two doctor parts

Lines that are not pure moves, each required by a gate or rule: the product name is respelled "Kiro Crew" in one moved comment and three moved docstrings, and eight moved data lines that must keep KiroCrew get a # brand-ok marker; one stderr fixture's tool namespace is re-rooted to Example::Cli:: (the assertions read only FindupException); one docstring drops an untrue parenthetical about which builtins ship; the subprocess.run that starts the moved TestInstallPidfdChildWatcher child gains encoding="utf-8", errors="replace" (decoding the CPython child's output as UTF-8), so its subprocess-encoding row goes; own-line # nosemgrep: / # flake-ok: comments sit above moved findings, and four kept TestDoctor fixtures get one too because test_cli.py parses under semgrep.

Backwards compatibility

Compatible: test-only change. Every collected test keeps its class, name, parameters and markers (the multiset is identical before and after). No test module imports test_cli, and no production module, import path, patch target or CLI changes.

Removes nothing: every test, fixture and helper still exists.

Tests

No test was added or rewritten. These proofs show the move lost nothing:

Check Result
pytest --collect-only before part 1 (207ed5ae4c: test_cli.py, plus the three TestCronCli items #15524 added since) vs HEAD (test_cli.py + its 12 parts) 541 = 541; the (class, name, params) multiset is identical (the consent-surface class from part 1 mapped to its BASE class)
pytest --fixtures-per-test before part 1 vs HEAD identical fixture set for all 541 tests
item markers before part 1 vs HEAD identical (541)
the 13 family files, the related set without the files that only name conftest.py (60), every corpus gate scripts/leaf_test_scope.py derives (608), every source_corpus / repo_files( scanner, the facade patch scanners and every test file main changed since part 1's base: 715 files, -n 3 (at the first cut of this commit, on main 64c6f446e3; the head differs from it only in docstrings, two marker reasons, one conftest.py docstring cite and the #15524 lines in test_cli_cron.py) 40799 passed, 126 skipped, 5 xfailed
at the head: the family, the facade scanners, both ratchets, test_file_line_cap.py, test_host_isolation_floor_restores.py and the cite files (36), the related set (61) and the 123 corpus scanners main changed since the first cut: 192 files, -n 3 9938 passed, 2 skipped
the five files with non-move edits (test_cli_setup.py, test_cli_main_entry.py, test_cli_gateway_client.py, test_cli_restart.py, test_cli_mcp_doctor.py), -n0, three seeded shuffled orders 204 passed, 2 skipped each time
test_cli_cron.py, which holds the ported #15524 tests, -n0, five seeded shuffled orders 30 passed each time
the re-rooted stderr fixture: with _doctor_mcp_tools mutated to drop the indented stderr lines test_failure_shows_error_head_and_indented_stderr fails (and passes unmutated)
semgrep 1.78, CI configs, --baseline-commit main, over the 16 changed .py files 0 findings; test_cli.py parses (none of its classes holds a bare-* lambda); with --disable-nosem exactly the marked sites
static floor: isort (CI command), black, flake8 (CI command), black baseline, subprocess encoding, comment history, brand, docs-lint, line cap, testpaths, feature map, harness parity, cwd reads, agent-sdk boundary, memory-store seam, focus cue all exit 0

Move audit

test/test_cli.py: 9533 -> 621 lines; moved 96.85% (python-ast)
   changed: TestResolveClientPortRunMarker [ClassDef] 127 lines    (two docstrings: "Kiro Crew")
   changed: TestInstallPidfdChildWatcher._install_then_spawn_in_child [FunctionDef] 59 lines    (encoding="utf-8", errors="replace")
   changed: TestDoctorMcpTools.test_failure_shows_error_head_and_indented_stderr [FunctionDef] 35 lines    (stderr fixture namespace re-rooted)
   changed: TestDoctorMcpTools.test_only_managed_servers_are_probed [FunctionDef] 31 lines    (docstring: "Kiro Crew's")
   changed: TestMcpBuiltinDispatch [ClassDef] 26 lines    (docstring: drops an untrue parenthetical)
   changed: Expr [Expr] 6 lines    (module docstring: the paragraph naming what stays and the parts)

Every other unit moved with an identical AST; comment-only edits (the respelled TestRestart comment, # brand-ok, # nosemgrep:, # flake-ok:) leave the AST unchanged. git diff main...HEAD is 829,921 bytes. Main's #15524 additions to _add_job_kwargs and TestCronCli move with those units into test_cli_cron.py, byte-identical to main.

Moved findings, each with an own-line marker: # nosemgrep: kirocrew.test-autouse-shared-monkeypatch on five moved class fixtures (between decorator and def); # nosemgrep: kirocrew.mock-pid-allocatable on nine MagicMock(pid=N) doubles that are never signalled; # nosemgrep: kirocrew.test-stdlib-clock-rebound plus # flake-ok: on the five time.sleep stubs; # flake-ok: on the loopback server's shutdown, join and three 5 s socket ceilings, the child-watcher child's 60 s bound and the in-process proc.wait(). test_cli.py itself parses under semgrep 1.78 (the bare-* lambdas moved with their classes), so its four kept TestDoctor autouse fixtures carry the same marker. Placeholder paths (/home/u/..., /Users/x/.toolbox/...), the @builder-mcp server name, the "Dev/Brazil-workspace" comment and amazon_transcribe module names move verbatim; each has merged added-line precedent.

Diff alignment (L36): git diff and git diff --minimal give the same added/deleted counts for every changed file: test_cli.py 13/8925, cli_helpers.py 114/5, cli.md 2/2, conftest.py 2/2, test/conftest.py 1/1, each baseline 0/1, each cite 1/1, and each new file only added lines.

Logger names (L25): no new module defines a logger.

Citations re-pointed (L29, same line count), file:line old -> new:

  • conftest.py:2434: test_cli.py -> test_cli_main_entry.py (the file whose cli.main() tests leak the log listener)
  • conftest.py:2575: test_cli.py::TestCronCli::test_cli_argparse_cron_add_agent_flag -> test_cli_cron.py::TestCronCli::...
  • test/conftest.py:1757: test_cli.py::test_real_subprocess_works_after_install_on_linux -> test_cli_main_entry.py::...
  • docs/system-specs/modules/cli.md:1279: test_cli.py::TestSeedDispatch::... -> test_cli_main_entry.py::TestSeedDispatch::...
  • docs/system-specs/modules/cli.md:1762: test/test_cli.py::TestDesktopGatewayIdentityParity -> test/test_cli_stop.py::TestDesktopGatewayIdentityParity
  • test/test_cli_server_facade.py:275: test_cli.py swaps -> test_cli_restart_readiness.py and test_cli_token.py swap
  • test/test_cli_split_dispatch.py:6: test_cli.py's lock-refusal and --seed tests -> test_cli_main_entry.py's

References not edited (cost): website/electron/gateway-stop.js:34 names test/test_cli.py::TestDesktopGatewayIdentityParity, which lives in test/test_cli_stop.py; tracked as a row on #21320 (#21320 (comment)).

Kept verbatim on purpose: the moved _pin_default_config docstring (like part 1's _doctor_reads_the_warm_probe_cache) says "this file"; cli_helpers.py's module docstring says that means test_cli.py and its doctor parts.

CI path references

None: no workflow filter, explicit file list, AUTOSDE.yaml glob or sensitive-change glob names test_cli.py.

Manual verification

N/A: unit coverage is sufficient. This is a test-only move, and the collected-test multiset plus the runs of every moved test prove it.

Related Issues

Part of #20192

Follow-up: #20383 (pre-existing shapes moved verbatim; de-flake there; this PR's sites are listed in #20383 (comment))

Follow-up: #21320 (deferred comment citation in website/electron/gateway-stop.js)

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@bolichen97
bolichen97 requested review from a team and KiroCrewCaptain as code owners October 19, 2026 11:27
@bolichen97
bolichen97 requested a review from mbriones98 October 19, 2026 11:27
@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision bfef27bb3509303ea5ead629ed9474aafd534cf9 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision bfef27bb3509303ea5ead629ed9474aafd534cf9 ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Advisory — does not block merge.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 19, 2026
@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of bfef27bb3509303ea5ead629ed9474aafd534cf9 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

website/electron/gateway-stop.js:34 still points to test/test_cli.py::TestDesktopGatewayIdentityParity, but that class now lives in test_cli_stop.py; re-point the citation.

[DESIGN-REVIEWED] bfef27b

Description read

[DESCRIPTION-READ] e4a833cdbe7a3e88c73807cc8844edc6e7b71fdda8ee134309a3d89d6f60062e

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed bfef27bb3509303ea5ead629ed9474aafd534cf9 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] bfef27b

Verdict parsed from the review's SHA-scoped output markers for commit bfef27bb3509303ea5ead629ed9474aafd534cf9.

False positive or not applicable? A repository writer can comment:
/ai-review override fable bfef27bb3509303ea5ead629ed9474aafd534cf9: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of bfef27bb3509303ea5ead629ed9474aafd534cf9 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] bfef27b

False positive or not applicable? A repository writer can comment:
/ai-review override gpt bfef27bb3509303ea5ead629ed9474aafd534cf9: <one-sentence reason>

@bolichen97
bolichen97 enabled auto-merge (squash) October 19, 2026 11:35
@bolichen97

Copy link
Copy Markdown
Contributor Author
  • Doctor parts still pull in _doctor_reads_the_warm_probe_cache: confirmed at the head

Confirmed at 4adfaf2: test_cli_mcp_doctor.py:15 and test_cli_stt_memory_doctor.py:15 import the autouse fixture from cli_helpers (as test_cli.py:19 and every other part do).
pytest --fixtures-per-test over test_cli.py and its 12 parts at this head lists the fixture for all 538 collected tests, and the per-test fixture sets equal those of test_cli.py before part 1 (207ed5a).
Outcome: no change needed.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 19, 2026

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REQUEST_CHANGES at 4adfaf2.

The split itself is sound. Every test and helper of the base test_cli.py appears AST-identical in the 12 family files, apart from the three edits the description declares (a brand respelling, the re-rooted FindupException fixture and the UTF-8 decode on one subprocess.run). All AI lanes pass, the Design punchline is answered, and every check on this head is green.

One deferred item is not tracked yet. The description defers the stale citation at gateway-stop.js#L33-L34, which still names test/test_cli.py::TestDesktopGatewayIdentityParity, to #21320. That issue's table and comments have no row for it. Add a row to #21320 (it should say test/test_cli_stop.py::TestDesktopGatewayIdentityParity, moved by #22152), or fix the comment in this PR.

I compared sources by AST; I did not run the tests.

— crew-pr-reviewer (r112)

@bolichen97

Copy link
Copy Markdown
Contributor Author

@iamwhatever thanks for the review (r112 at 4adfaf2).

Deferred citation in website/electron/gateway-stop.js:33-34: tracked. #21320 now has the row: #21320 (comment) (gateway-stop.js:34: test/test_cli.py::TestDesktopGatewayIdentityParity -> test/test_cli_stop.py::TestDesktopGatewayIdentityParity, moved by #22152; the class is at test/test_cli_stop.py:270).

The JS comment stays out of this PR on purpose: editing a website/ file would pull the frontend lanes and a screenshot waiver onto a test-only backend split, so comment-only website citations are batched into the one docs PR that #21320 tracks. No code change, so no push.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: passed Eligible automated validation passed for the current revision labels Oct 19, 2026
iamwhatever
iamwhatever previously approved these changes Oct 20, 2026
@github-actions github-actions Bot added merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention labels Oct 20, 2026
mbriones98
mbriones98 previously approved these changes Oct 20, 2026

@mbriones98 mbriones98 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checks green and readiness passed at 4adfaf2; no behavior change.

test/test_cli.py is about 9,500 lines after part 1, still over the
2,000-line cap. This second PR moves its command tests into one file
per command or concern (cron, setup, main entry, gateway client, stop,
restart, restart readiness, token, the doctor's MCP and STT/memory
sections) and the doctor tests' helpers into test/cli_helpers.py.
test_cli.py keeps the doctor's full report, the update failure paths,
the banner spelling and the config-set grammar, and its line-cap row
goes.

Test bodies move verbatim, except lines a gate requires: the product
name respelled in one comment and three docstrings, brand-ok markers on
eight data lines, one stderr fixture's namespace re-rooted, one
docstring reworded to name no internal process, an explicit UTF-8
decode on the moved child-watcher subprocess (its subprocess-encoding
row goes), and own-line nosemgrep / flake-ok markers above moved
findings. The collected (class, name, params) multiset, fixtures per
test and markers equal BASE.

Part of #20192
@bolichen97
bolichen97 dismissed stale reviews from mbriones98 and iamwhatever via bfef27b October 20, 2026 02:09
@bolichen97
bolichen97 force-pushed the refactor/linecap-t59-test-cli-p2 branch from 4adfaf2 to bfef27b Compare October 20, 2026 02:09
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision labels Oct 20, 2026
@github-actions github-actions Bot removed the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Oct 20, 2026
@bolichen97

Copy link
Copy Markdown
Contributor Author

The cite is a comment in a frontend file. This PR changes only backend tests, their helper module, two baseline rows and backend cites. Editing website/** would pull the frontend and screenshot lanes onto a test-only diff for a comment-only fix.
The stale cite is listed under "References not edited (cost)" in the description with its new home, test/test_cli_stop.py. It is a row on #21320 (#21320 (comment)), the tracker for one batched docs PR that re-points every such cite after the splits land; the reviewer accepted that on this PR (#issuecomment-6269509675).
Nothing reads the comment at run time, and docs/system-specs/modules/cli.md:1762 already names test/test_cli_stop.py::TestDesktopGatewayIdentityParity.
Outcome: rebutted for this PR (no change here); the re-point lands with #21320.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 20, 2026

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE at bfef27b.

The split moves tests and changes nothing they check. I compared every top-level unit and class member of the base test_cli.py and cli_helpers.py by AST against test_cli.py, cli_helpers.py and the 11 new part files at this head. Every unit matches, apart from module docstrings and the five edits the description declares: two "Kiro Crew" respellings, the re-rooted FindupException fixture, one reworded docstring, and the UTF-8 decode on one subprocess.run. The only other edits re-point citations and remove two baseline rows. No test_cli.py:: node id remains outside website/. The gateway-stop.js cite that r112 asked about is now a row on #21320, and the Design punchline has a current-head disposition. The GPT and Opus lanes found nothing, and every check on this head is green. I did not run the tests.

— crew-pr-reviewer (r114)

@iamwhatever
iamwhatever merged commit 2c20a76 into main Oct 20, 2026
106 of 107 checks passed
@iamwhatever
iamwhatever deleted the refactor/linecap-t59-test-cli-p2 branch October 20, 2026 05:16
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants