Repository navigation
test: split test_ai_review_workflows.py, part 2 of 3 - #20347
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing to check. [DESIGN-REVIEWED] 1961e61 Description read[DESCRIPTION-READ] 53cb6c35718b7c2a8d9372593ee12774a52cf5eab7abea4ed69a02d5ba72ec72 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Check one thing before merge: CI's collected count still matches base (1,489 tests). The seven pins moved into What this change shipsInventory (6 items) — 6 justifiedIntent: Bring the review-workflow test file under the documented 2,000-line cap by moving tests verbatim. This is a refactor: neither a fix nor an addition.
[FIRST-PRINCIPLES-REVIEWED] 1961e61 Description read[DESCRIPTION-READ] f50a831751c6d67b026c90839983481b2361f85282e073f37e11a195eab901d1 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
|
Security Scope Review -- ✅ PASSWhich legitimate operations does Verdict basis: adjudicated, zero confirmed regressions. Security scope review -- adjudicated candidates
No candidate flipped from allowed to refused. Scope-Verdict: PASS TIGHTENING: none -- the patch makes nothing stricter, so there is no threat to quote. It touches no file under ADJUDICATED: 8 candidates, 0 confirmed newly-refused PLATFORM COVERAGE: the patch assumes no path, separator, interpreter, executable name or environment variable in any guard. The candidates still probe the maintainers' daily operations in the touched area, in both spellings:
No platform spelling the diff assumes is left unprobed. [SCOPE-REVIEWED] 1961e61 |
6d7d7d9 to
54792d7
Compare
|
mbriones98
left a comment
There was a problem hiding this comment.
Checks green and readiness passed at 54792d7; no behavior change.
54792d7 to
ed5d415
Compare
|
|
|
HOLD at ed5d415. The split looks sound. I compared removed and added lines in the diff: every moved line reappears verbatim, the 192 test definitions match one for one, and the only non-move edits are the line-cap baseline row and comment lines that now name What holds it: the move rehomes the credential-isolation pins into test_ai_review_workflows_credential_isolation.py and edits comments in the fork and security-scope review workflows. Changes to credential and security files are reserved for human review, so a maintainer needs to sign off. The workflow edits are comment-only and add no permissions, secrets or token exposure. This clears when a maintainer reviews and approves the move. I read the diff and CI results; I did not run the tests. — crew-pr-reviewer (r102) |
The second part of the split moves the run from the block-adjudication tests through the fork review sandbox, and the file's tail from the guarded-upsert outcome tests on (28 classes, 596 tests), into seven new sibling parts and the parts part 1 started. TestReviewLaneVerdictVisibility is 1,549 lines, and two classes drive it as their harness, so the three cannot share a 1,600-line part. Its seven workflow-source pins move to a new topic class, TestReviewLaneVerdictVisibilityAcrossLanes, in test_ai_review_workflows_comment_writes.py; the harness and its behavioural tests keep the class name the harness users bind. Test bodies move verbatim; only those seven tests change class. The workflow and doc comments that cite the byte-identity pins name their new file. Part of #20192
ed5d415 to
1961e61
Compare
|
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Approved after a maintainer read the review lanes on this head: all clean; held only on the file count of a 1:1 test split.
Problem / Motivation
Goal: Bring
test/test_ai_review_workflows.pyunder the 2,000-line cap on hand-written files without changing a single test, in a three-PR series; this PR is part 2.test/test_ai_review_workflows.pyis still 13,971 lines after part 1 (#20282). It is past the 2,000-line cap on hand-written files, and nearly every review-workflow change collides in it.Why it matters
Every change to a review lane edits this one file, so unrelated PRs conflict in it and a reader has to search thousands of lines for the pin that guards a workflow. Splitting by the behaviour under test gives each lane's pins a findable home.
Not a goal
What changed (motivation → approach → change)
Like part 1, this part moves ONE contiguous run of the original, so git's default diff equals
git diff --minimalbyte for byte (701,226 bytes). The run isTestBlockAdjudicationContractthroughTestReviewSandboxFailsClosed, plus the file's tail fromTestTheGuardedUpsertRecordsWhatReachedTheSloton, which part 1 left adjacent to it: 28 classes, 596 tests.test/test_ai_review_workflows_block_adjudication.py(new)test/test_ai_review_workflows_verdict_publication.py(new)test/test_ai_review_workflows_whole_design_verdicts.py(new)test/test_ai_review_workflows_gpt.py(new)test/test_ai_review_workflows_first_principles.py(new)test/test_ai_review_workflows_human_override.py(new)test/test_ai_review_workflows_code_review_greps.py(new)test/test_ai_review_workflows_comment_writes.pyTestReviewLaneVerdictVisibilityAcrossLanes)test/test_ai_review_workflows_credential_isolation.pytest/test_ai_review_workflows_fork_verdict.pytest/ai_review_workflows_helpers.py_GUARDED_LANESand the helpers the new parts share with the originalTestReviewLaneVerdictVisibilityis 1,549 lines, and two classes instantiate it as their harness, so the three must share a file: 1,729 lines, over the 1,600-line target. Its seven workflow-source pins (byte-identical guard and write primitive, retried lookups, honest write reports) move to a new topic class,TestReviewLaneVerdictVisibilityAcrossLanes, incomment_writes; the harness and its behavioural tests keep the class name the two harness users bind, inverdict_publication. Those seven tests are the only ones whose class changes.The comments that cite those byte-identity pins (12 review workflows, the CI doc and two tests) now name
test_ai_review_workflows_comment_writes.py.Backwards compatibility
Compatible: only test modules move; no product import path, entry point or API changes, and every test keeps its name and parameters.
Removes nothing: the 596 moved tests still run, from sibling files in the same directory, and nothing imports from
test/test_ai_review_workflows.py.Move audit
Tests
No test is added or changed. The proof that nothing was lost:
TestReviewLaneVerdictVisibilityClass mapping (29 collected tests):
TestReviewLaneVerdictVisibility::test_guard_function_is_byte_identical_across_all_lanesTestReviewLaneVerdictVisibilityAcrossLanes::test_guard_function_is_byte_identical_across_all_lanesTestReviewLaneVerdictVisibility::test_only_the_fail_closed_lanes_exit_on_a_verdict_that_missed_the_slotTestReviewLaneVerdictVisibilityAcrossLanes::test_only_the_fail_closed_lanes_exit_on_a_verdict_that_missed_the_slotTestReviewLaneVerdictVisibility::test_the_lookup_that_gates_a_verdict_write_is_retriedTestReviewLaneVerdictVisibilityAcrossLanes::test_the_lookup_that_gates_a_verdict_write_is_retriedTestReviewLaneVerdictVisibility::test_write_primitive_is_byte_identical_across_every_publishing_laneTestReviewLaneVerdictVisibilityAcrossLanes::test_write_primitive_is_byte_identical_across_every_publishing_laneTestReviewLaneVerdictVisibility::test_no_verdict_write_announces_a_success_it_did_not_getTestReviewLaneVerdictVisibilityAcrossLanes::test_no_verdict_write_announces_a_success_it_did_not_getTestReviewLaneVerdictVisibility::test_only_the_idempotent_write_may_repeat_without_confirmingTestReviewLaneVerdictVisibilityAcrossLanes::test_only_the_idempotent_write_may_repeat_without_confirmingTestReviewLaneVerdictVisibility::test_every_lane_calls_the_guard_and_fork_fp_covers_both_sitesTestReviewLaneVerdictVisibilityAcrossLanes::test_every_lane_calls_the_guard_and_fork_fp_covers_both_sitestest_ai_review_workflows*file, the tests that read an edited workflow, the 21 that scan every workflow, and the doc/ratchet readers): 3,668 passed.git diff --numstatandgit diff --minimal --numstatagainst the base agree (the original: +8 / -6,612).test/test_flake_pattern_ratchet.py,test/test_wait_discipline_ratchet.py, comment-history and brand gates, black, isort, flake8,docs_lint.py, semgrep 1.78 with CI's configs (0 findings): all pass.Determinism: the moved tests are unchanged, so their determinism is what it was on main.
CI path references
Comment lines only, no logic; each names the file that now holds
TestReviewLaneVerdictVisibilityAcrossLanes:test_ai_review_workflows.pypins it byte-identical across all" (the write primitive) ->test_ai_review_workflows_comment_writes.py, inclaude-review.yml,design-review.yml,first-principles-review.yml,fork-design-review.yml,fork-first-principles-review.yml,fork-gpt-review.yml,fork-opus-review.yml,fork-security-scope-review.yml,fork-ux-review.yml,security-scope-review.yml,ux-review.yml.test_ai_review_workflows.py" (guarded_comment_upsert) ->test_ai_review_workflows_comment_writes.py, in the same files exceptclaude-review.yml(10 files).codex-review.yml: "test_ai_review_workflows.pypins both byte-identical across all lanes" (its guarded upsert and write primitive) ->test_ai_review_workflows_comment_writes.py.No workflow path filter or file list names the target.
Citations updated
docs/ci/ci-and-reviews.md: the guarded-upsert byte identity ->test_ai_review_workflows_comment_writes.py; names the file now holdingTestOverrideHandlerReRunsTheBoundForkLaneRun.test/test_review_slot_read_error.py(docstring and one comment) andtest/test_withheld_verdict_retention.py(one comment): ->test_ai_review_workflows_comment_writes.py. Comment text only.Moved literals the content scan reads
AKIAIOSFODNN7EXAMPLE(AWS's documented example id, 2 lines) moves withTestBlockAdjudicationArithmetictoblock_adjudication.py:941,949; the bareFAKE_SECRET = "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"(the documented example secret) moves withTestForkLaneRedactsCredentialValuestocredential_isolation.py:292;"ghp_" + ...is built at runtime.Line-cap baseline
Lowers this target's row in
.github/file-line-cap-baseline.txtfrom 14,106 to 7,367 (it stays over 2,000 until part 3).python3 scripts/check_file_line_cap.py: 0 failures.Upstream change carried
#20332 (codex-review.yml converges onto the shared guarded upsert) landed after this PR opened and edited the classes it moves. The recipe was re-run on main
ec99d9056b, so its edits move verbatim:TestGptVerdictVisibilityandTestGptRefusalTerminalStateingpt,TestReviewLaneVerdictVisibilityand its two harness users inverdict_publication, the seven pins incomment_writes, and the new codex row of_GUARDED_LANESin the helper module.Notes for reviewers
TestReviewLaneVerdictVisibilityAcrossLanesincomment_writes.gpt,whole_design_verdicts,first_principles,human_override) describe what they hold now; part 3 widens them.Manual verification
N/A — test-only move; the collected multiset, the move audit and the passing runs above are the verification.
Related Issues
Part of #20192
Follow-up: #20383 (pre-existing shapes moved verbatim; de-flake there)
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)