Skip to content

test: tighten stale comments in session tests other prs touch - #37334

Merged
iamwhatever merged 1 commit into
mainfrom
crew/testoptimizer/test-quality-catchup
Nov 14, 2026
Merged

iamwhatever merged 1 commit into
mainfrom
crew/testoptimizer/test-quality-catchup

Conversation

@iamwhatever

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Make the comments in the session test files that other open PRs touch short, true and useful, the same pass the rest of the session tests got, with no change to test behavior.

Three session test files carry review-round section headers ("GPT 6.1 current-head findings 1-5", "fork-lane review, second round") and Regression: openers that describe the old bug in past tense instead of the rule the test holds.

Why it matters

A reader needs to know what each group of tests covers and what must hold. Review-round names and past-tense bug stories say neither.

Not a goal

  • No change to any test's code, names, assertions, fixtures or assertion messages (the test_gpt_N_* names stay).
  • No new comments that restate the code.
  • Security-held test files.

What changed (motivation → approach → change)

Before After
"# ── GPT 6.1 current-head findings 1-5 ──" "# ── block boundaries: raw HTML, fences, lists ──"
"# ── fork-lane review (GPT 6.1) findings: three more ways a turn could hide the rest ──" "# ── quoted comments and other ways a turn could hide the rest ──"
"# ── fork-lane review, second round ──" (an empty section) removed
"Regression: float(10**400) raises OverflowError, which was uncaught." "float(10**400) raises OverflowError; the handler must catch it."
"Regression: ... the bug that flipped the pill from the true 41,336/10,000 (413%) to a misleading 10,000/10,000" "... that would flip the pill from the true 41,336/10,000 (413%) to 10,000/10,000 (100%, overage hidden)."

Three files (test_session_markdown.py, test_session_autocompact_override.py, test_session_usage.py): 19 insertions, 23 deletions. goal_scan.py astcheck against the merge-base: files checked: 3 code-changed: 0 docstring-only: 0. Open PRs #36873, #36630 and #34722 touch these files; the change is comment-only, so either side rebases cleanly.

Backwards compatibility

Compatible: test-only change, comments only.

Removes nothing: no code, key, path, record or function is removed or renamed.

Tests

Proof: not provable -- comment-only change; the AST check shows no code difference

The three touched files plus test/test_flake_pattern_ratchet.py pass on this branch (pytest -n 2 -p no:randomly: 386 passed). flake8, check_comment_history.py and check_brand_name.py pass; no added line is over 100 columns.

Manual verification

N/A — comment-only change.

Related Issues

N/A — part of the test comment quality pass.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license.

Rewrite review-round section headers and Regression: openers in the
session markdown, autocompact override and usage tests as present-tense
rules. These files were skipped in the session pass because other open
PRs touch them. Comments only; no test behaviour changes.
@iamwhatever
iamwhatever requested review from a team and KiroCrewCaptain as code owners November 14, 2026 19:59
@iamwhatever
iamwhatever enabled auto-merge (squash) November 14, 2026 19:59
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 14, 2026
@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision 4f6fddc447cfd9c8797224f334502113abd715ed ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 4f6fddc447cfd9c8797224f334502113abd715ed touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 4f6fddc447cfd9c8797224f334502113abd715ed — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Comment-only test change: no code, behavior or reader is affected, so nothing blocks.

[DESIGN-REVIEWED] 4f6fddc

Description read

[DESCRIPTION-READ] 37e5236b356f09562eeb221011df46f3b2024655a5484943c9bf08f05037451d

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 4f6fddc447cfd9c8797224f334502113abd715ed — this comment is updated in place on each push.

Review details

Comment-only changes in tests; no candidates.

No findings.

[OPUS-REVIEWED] 4f6fddc

Verdict parsed from the review's SHA-scoped output markers for commit 4f6fddc447cfd9c8797224f334502113abd715ed.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 4f6fddc447cfd9c8797224f334502113abd715ed false-positive: <reason, 20 to 500 characters>

@github-actions

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 4f6fddc447cfd9c8797224f334502113abd715ed and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 4f6fddc

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 4f6fddc447cfd9c8797224f334502113abd715ed false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 14, 2026

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. On head 4f6fddc the change is comments only in three test files. Each file parses to the same AST at the merge-base and at the head. A diff that ignores whitespace and comments leaves only two removed blank lines. PR Readiness passed on this head. The new section headers match the tests under them. The rewritten comments are accurate, for example the OverflowError handling in _validate_autocompact_pct. The hunks do not collide with the open PRs that touch the same files.

Done well: past-tense "Regression: X was uncaught" stories now state the rule each test holds, and review-round labels became names for what the tests check.

Optional:

  • test/test_session_markdown.py:1137 still has the review-round header "current-head scanner-state regressions", and :1434 says "The other direction of F1". Rewording them to state the rule would finish this file. A follow-up in the same series is fine.

@iamwhatever
iamwhatever merged commit 3e49832 into main Nov 14, 2026
97 checks passed
@iamwhatever
iamwhatever deleted the crew/testoptimizer/test-quality-catchup branch November 14, 2026 21:48
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants