Repository navigation
test: split test_ai_review_workflows.py, part 3 of 3 - #20654
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Check that [DESIGN-REVIEWED] 5861761 Description read[DESCRIPTION-READ] 8da986b46474cc42c3e34c99fefa8a14162cf083a308df9109243683abfa30da That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Check that CI collects the same 1,501 tests on this head. The description claims parity, but the claim is all it gives. What this change shipsInventory (6 items) — 6 justifiedIntent: bring
[FIRST-PRINCIPLES-REVIEWED] 5861761 Description read[DESCRIPTION-READ] c3d833ab8de4a9aca9919fe39b69d4fcc7c7ab82e1d70ab34c796cb83a901d81 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
The last part of the split moves everything between the module helpers and TestPrReadiness, and the run from the First Principles shell-syntax tests through the protected check-name tests (30 classes, 428 tests). Three new sibling parts take them, beside five parts that parts 1 and 2 started. No file is now over 1,600 lines. The original keeps TestPrReadiness, which another test reads by path, and the two classes beside it: TestFirstPrinciplesReview holds an unsplit PEM header literal the content scan has no precedent for, and TestDesignReviewPresentation keeps the kept run contiguous. TestUxReviewReadsTheScreenshotsBlindFirst is 2,298 lines, so it is cut into three topic classes (Pipeline, DescriptionEvidence, CommittedEvidence). The evidence-gate harness both evidence classes drive moves once into ai_review_workflows_helpers.py as UxEvidenceGateHarness, a base class that holds no test. Test bodies move verbatim; only the UX tests change class. The workflow, doc and test comments that cite the moved pins name their new files. Part of #20192
70bacad to
5861761
Compare
|
|
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Approved by a maintainer after reading the review lanes: every lane is clean on this head, and the automated triage held it only on a size, path, category or doc rule.
Problem / Motivation
Goal: Bring
test/test_ai_review_workflows.pyunder the 2,000-line cap on hand-written files without changing a single test, in a three-PR series; this PR is part 3.test/test_ai_review_workflows.pyis still 7,367 lines after parts 1 and 2 (#20282, #20347). It is past the 2,000-line cap on hand-written files, and nearly every review-workflow change collides in it.Why it matters
Every change to a review lane edits this one file, so unrelated PRs conflict in it and a reader has to search thousands of lines for the pin that guards a workflow. Splitting by the behaviour under test gives each lane's pins a findable home.
Not a goal
What changed (motivation → approach → change)
Like parts 1 and 2, this part moves ONE contiguous run of the original, so git's default diff equals
git diff --minimalbyte for byte (679,041 bytes; per file,git diff --numstatand--minimal --numstatagree, the original at +1 / -6,374). It moves everything between the module helpers andTestPrReadiness, and the run fromTestFirstPrinciplesShellSyntaxthroughTestProtectedCheckNameHasOnePublisherPerPrType: 30 classes, 428 tests. The original ends at 1,247 lines and keeps one contiguous run:TestPrReadiness(whichtest/test_review_lane_prompt_parity.pyreads by path),TestDesignReviewPresentation, andTestFirstPrinciplesReview, which holds an unsplit PEM private-key header as a case literal that the content scan has no merged precedent for, so it is not moved.test/test_ai_review_workflows_ux.py...Pipeline(the blind pass) and...DescriptionEvidencetest/test_ai_review_workflows_ux_committed_evidence.py(new)...CommittedEvidencetest/test_ai_review_workflows_opus.py(new)test/test_ai_review_workflows_fork_stage2.py(new)test/test_ai_review_workflows_human_override.pytest/test_ai_review_workflows_gpt.pytest/test_ai_review_workflows_first_principles.pytest/test_ai_review_workflows_whole_design_verdicts.pytest/ai_review_workflows_helpers.pyUxEvidenceGateHarnessand the helpers the new parts share with the originalTestUxReviewReadsTheScreenshotsBlindFirstis 2,298 lines, over the 1,600-line cap for one part, so it is cut into three topic classes:...Pipeline(the blind pass and its steps) and...DescriptionEvidenceinux, and...CommittedEvidenceinux_committed_evidence. The evidence-gate harness both evidence classes drive (fixture bytes, the gh/jq/curl stubs, the git fixture repository and the step runner) moves once intotest/ai_review_workflows_helpers.pyasUxEvidenceGateHarness: noTestprefix and no test method, so pytest collects nothing from it, and both evidence classes inherit it. The three-line_stepsaccessor is copied into the two topic classes that call it.The workflow, doc and test comments that cite the moved pins name their new files.
Backwards compatibility
Compatible: only test modules move; no product import path, entry point or API changes, and every test keeps its name and parameters.
Removes nothing: the 428 moved tests still run, from sibling files in the same directory, and nothing imports from
test/test_ai_review_workflows.py.Move audit
Tests
No test is added or changed. The proof that nothing was lost:
TestUxReviewReadsTheScreenshotsBlindFirstClass mapping (57 collected tests, by method;
[...]is every parametrization):TestUxReviewReadsTheScreenshotsBlindFirst::)test_the_fork_lane_reads_the_media_a_no_write_access_contributor_commitsTestUxReviewReadsTheScreenshotsBlindFirstCommittedEvidencetest_a_committed_symlink_is_refused_by_its_mode_before_its_bytes_are_readTestUxReviewReadsTheScreenshotsBlindFirstCommittedEvidencetest_rejected_committed_media_still_consumes_the_blob_read_budgetTestUxReviewReadsTheScreenshotsBlindFirstCommittedEvidencetest_a_description_attachment_is_never_displaced_by_a_committed_fileTestUxReviewReadsTheScreenshotsBlindFirstCommittedEvidencetest_the_committed_script_runs_under_the_bash_3_of_macosTestUxReviewReadsTheScreenshotsBlindFirstCommittedEvidencetest_a_glob_metacharacter_in_a_committed_name_names_that_one_entryTestUxReviewReadsTheScreenshotsBlindFirstCommittedEvidencetest_a_committed_name_with_a_control_character_reaches_no_sinkTestUxReviewReadsTheScreenshotsBlindFirstCommittedEvidencetest_a_committed_format_the_reviewer_cannot_read_is_named_not_droppedTestUxReviewReadsTheScreenshotsBlindFirstCommittedEvidencetest_a_refused_committed_format_consumes_no_read_budgetTestUxReviewReadsTheScreenshotsBlindFirstCommittedEvidencetest_a_screenshot_that_was_only_moved_is_named_as_the_base_not_admittedTestUxReviewReadsTheScreenshotsBlindFirstCommittedEvidencetest_a_failed_enumeration_fails_the_step_instead_of_reading_as_no_mediaTestUxReviewReadsTheScreenshotsBlindFirstCommittedEvidencetest_both_fork_lanes_read_the_committed_media_from_the_object_storeTestUxReviewReadsTheScreenshotsBlindFirstCommittedEvidencetest_the_same_repo_design_lane_sources_the_shared_script_and_keeps_no_loop_of_its_ownTestUxReviewReadsTheScreenshotsBlindFirstCommittedEvidencetest_both_design_lanes_admit_a_renamed_screenshot_and_refuse_by_mode_and_bytesTestUxReviewReadsTheScreenshotsBlindFirstCommittedEvidencetest_a_design_lane_attachment_is_counted_apart_from_the_committed_mediaTestUxReviewReadsTheScreenshotsBlindFirstCommittedEvidencetest_neither_fork_lane_tells_its_reviewer_to_discount_a_committed_imageTestUxReviewReadsTheScreenshotsBlindFirstCommittedEvidencetest_the_blind_pass_can_see_only_the_screenshotsTestUxReviewReadsTheScreenshotsBlindFirstPipelinetest_the_blind_pass_runs_before_the_review_and_feeds_it_a_data_fileTestUxReviewReadsTheScreenshotsBlindFirstPipelinetest_each_ux_model_call_has_its_own_credential_assumeTestUxReviewReadsTheScreenshotsBlindFirstPipelinetest_a_failed_blind_read_degrades_to_an_evidence_gap_not_a_red_laneTestUxReviewReadsTheScreenshotsBlindFirstPipelinetest_review_prompts_carry_no_expression_so_the_21000_cap_cannot_biteTestUxReviewReadsTheScreenshotsBlindFirstPipelinetest_the_five_second_proxy_is_replaced_not_duplicatedTestUxReviewReadsTheScreenshotsBlindFirstPipelinetest_the_fork_lane_says_it_has_no_blind_read_rather_than_faking_oneTestUxReviewReadsTheScreenshotsBlindFirstPipelinetest_each_case_gets_its_own_job_temp_not_the_shard_sTestUxReviewReadsTheScreenshotsBlindFirstDescriptionEvidencetest_the_evidence_step_copies_regular_images_under_opaque_namesTestUxReviewReadsTheScreenshotsBlindFirstDescriptionEvidencetest_the_evidence_step_reports_no_screens_for_a_code_only_ui_changeTestUxReviewReadsTheScreenshotsBlindFirstDescriptionEvidencetest_every_download_failing_fails_the_same_repo_evidence_stepTestUxReviewReadsTheScreenshotsBlindFirstDescriptionEvidencetest_the_evidence_step_reports_unfetched_false_when_nothing_was_offeredTestUxReviewReadsTheScreenshotsBlindFirstDescriptionEvidencetest_a_recording_only_description_is_missing_evidence_not_a_fetch_failureTestUxReviewReadsTheScreenshotsBlindFirstDescriptionEvidencetest_a_404_attachment_is_the_authors_url_not_a_fetch_failureTestUxReviewReadsTheScreenshotsBlindFirstDescriptionEvidencetest_a_partial_fetch_fails_the_step_like_a_total_oneTestUxReviewReadsTheScreenshotsBlindFirstDescriptionEvidencetest_all_four_evidence_lanes_admit_the_same_two_evidence_classesTestUxReviewReadsTheScreenshotsBlindFirstDescriptionEvidencetest_a_recording_that_downloaded_beside_a_failed_still_fails_the_stepTestUxReviewReadsTheScreenshotsBlindFirstDescriptionEvidencetest_the_fork_lane_fails_an_all_failed_fetch_and_reddens_its_check_runTestUxReviewReadsTheScreenshotsBlindFirstDescriptionEvidencetest_the_evidence_step_downloads_attachments_from_the_pr_descriptionTestUxReviewReadsTheScreenshotsBlindFirstDescriptionEvidencetest_a_transient_api_failure_is_retried_and_the_evidence_still_collectedTestUxReviewReadsTheScreenshotsBlindFirstDescriptionEvidencetest_three_api_failures_fail_the_step_closed_not_as_no_evidenceTestUxReviewReadsTheScreenshotsBlindFirstDescriptionEvidencetest_every_download_skipped_is_an_error_annotation_not_a_quiet_gapTestUxReviewReadsTheScreenshotsBlindFirstDescriptionEvidencetest_description_recordings_are_capped_and_never_count_as_screensTestUxReviewReadsTheScreenshotsBlindFirstDescriptionEvidencetest_description_images_share_the_committed_cap_and_downloads_are_boundedTestUxReviewReadsTheScreenshotsBlindFirstDescriptionEvidencetest_the_pr_description_reaches_the_evidence_step_only_as_grep_inputTestUxReviewReadsTheScreenshotsBlindFirstDescriptionEvidencetest_the_fork_lane_fetches_attachments_from_the_api_body_with_the_same_guardsTestUxReviewReadsTheScreenshotsBlindFirstDescriptionEvidencetest_both_lanes_name_the_description_as_the_normal_home_for_evidenceTestUxReviewReadsTheScreenshotsBlindFirstDescriptionEvidenceThe harness is not collected on its own: no node id names
UxEvidenceGateHarness.test_ai_review_workflows*file, the tests that read an edited workflow, doc or test, the tests that scan every workflow, and the doc/ratchet readers; 75 files): 4,899 passed, 10 skipped.test/test_flake_pattern_ratchet.py,test/test_wait_discipline_ratchet.py, comment-history and brand gates, black, isort, flake8,docs_lint.py, semgrep 1.78 with CI's configs (0 findings): all pass.Determinism: the moved tests are unchanged, so their determinism is what it was on main.
CI path references
Comment lines only, no logic: "
test_ai_review_workflows.pyasserts the guard is on every step" ->test_ai_review_workflows_fork_stage2.py(whereTestProtectedCheckNameHasOnePublisherPerPrTypenow lives), inclaude-review.yml,codex-review.yml,design-review.yml,first-principles-review.ymlandux-review.yml.No workflow path filter or file list names the target.
Citations updated
Eight comment and docstring citations (L29), each naming the part that now holds the pin it cites; per file,
git diff --numstatand--minimal --numstatagree:docs/ci/ci-and-reviews.md(2 edits)TestOpusDiscoveryCaptureExecutes->test_ai_review_workflows_opus.py; the override-handler pins ->test/test_ai_review_workflows_human_override.pydocs/system-specs/common/testing-conventions.mdTestUxReviewReadsTheScreenshotsBlindFirst*classes intest_ai_review_workflows_ux*.pytest/test_pr_quality_gates.py(two docstrings, one comment)test_ai_review_workflows_ux_committed_evidence.pyandtest_ai_review_workflows_ux*.pytest/test_fork_stage2_checkout.py(one comment)test_ai_review_workflows*.py; "pins" becomes "pin" to agree with the plural subjecttest/test_issue_triage_workflow.py(one comment)The five workflow comment lines are listed under
## CI path referencesabove (+1 / -1 each).Line-cap baseline
Deletes this target's row (
7367 test/test_ai_review_workflows.py) from.github/file-line-cap-baseline.txt: the file ends at 1,247 lines.python3 scripts/check_file_line_cap.py: 0 failures.Upstream changes carried
Three PRs edited the original after this PR's first push. The recipe was re-run on main
213339f671, so their text moves verbatim; the move audit and the collected multiset above are measured on that base.MERGE_BASEasserts inTestFirstPrinciplesReview; the UX evidence harness env;TestClaudeReviewCodeOnlyScopeTestFirstPrinciplesReviewstays in the original; the env inUxEvidenceGateHarness(helper module);TestClaudeReviewCodeOnlyScopeinopusTestOpusTwoStageArchitecture, the newTestOpusTurnBudgetOutcomeand_max_turns_transcriptopus, beside the class they extendNotes for reviewers
TestUxReviewReadsTheScreenshotsBlindFirstclass header: the class is cut into topic classes. Every member is AST-identical in its new class; the original class docstring stays on...Pipeline.Manual verification
N/A — test-only move; the collected multiset, the move audit and the passing runs above are the verification.
Related Issues
Part of #20192
Follow-up: #20383 (pre-existing shapes moved verbatim; de-flake there)
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)