Skip to content

test: drop history-only text from test docstrings - #24558

Merged
chenmingwei23 merged 1 commit into
mainfrom
chore/test-docstrings-history
Oct 27, 2026
Merged

chenmingwei23 merged 1 commit into
mainfrom
chore/test-docstrings-history

Conversation

@iamwhatever

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Test docstrings state the rule each test pins, without review-round history.

Many test docstrings open with "(review finding)", "Three review rounds each found...", "a reviewer caught...", commit SHAs or issue numbers. That text tells a reader how the test came to be, not what it guards, and it goes stale. This is part 1 of 2 (43 files, test/metrics/ through test/test_issue_radar_crew_runtime_sweep.py); part 2 covers the rest.

Why it matters

A reader of a failing test needs the invariant and why it matters. History narration hides that and is what scripts/check_comment_history.py already refuses on new lines.

Not a goal

  • No test code, assertion, fixture or test name changes.
  • No source (src/) changes.
  • Docstrings where review is the feature under test are left alone (listed below).

What changed (motivation → approach → change)

For each strict-rule history hit: cut the history; where the docstring also says what the test pins and why, kept that as the rule, stated in the present tense ("would ..." instead of "was ..."). No docstring that carried a real why was emptied. Long lines rewrapped to ≤100 columns.

Left as-is because review is the feature under test:

  • test/test_gh_pr_fetch.py:900, :976 (fetching PR review remarks)
  • test/test_fix_loop_discipline.py:5 (the reviewer-finding / deferral discipline itself)

Strict history count for test/ (goal_scan.py history --strict): 132 → 62 on this branch (70 lines cut or rewritten).

Backwards compatibility

Compatible: docstring text only; nothing is parsed from these strings.

Removes nothing: only docstring wording changes; no test, reader or behaviour is removed.

Tests

Proof: not provable -- test-only change

  • goal_scan.py astcheck 20c46e97d9 (merge-base):
files checked: 43  code-changed: 0  docstring-only: 43
  • All touched test files from both parts run once (-n 4, KIROCREW_PROFILE=standalone): 4893 passed, 16 skipped.
  • Collected test count over the touched files: 4909 at base and 4909 at head.
  • flake8, isort, black gate (check_black_formatting.py with black 26.10.1) and check_comment_history.py pass.

Manual verification

N/A — docstring-only change; AST check proves no code moved.

Related Issues

Part 2: chore/test-docstrings-history-part2.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license.

@iamwhatever
iamwhatever requested review from a team and KiroCrewCaptain as code owners October 26, 2026 20:27
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 26, 2026
@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 93a64097bb1e2a0199a7cac1475fb6ef6f62dccf touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision 93a64097bb1e2a0199a7cac1475fb6ef6f62dccf ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 93a64097bb1e2a0199a7cac1475fb6ef6f62dccf — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the diff only rewords docstrings in 43 test files. It adds no code, renders nothing new, and takes nothing away.

Notes

  • The ## Backwards compatibility section has both a Compatible: line and a Removes nothing: line. Keep only Removes nothing: so the section reads cleanly.

[DESIGN-REVIEWED] 93a6409

Description read

[DESCRIPTION-READ] 9150a552a27698337b03b2f90aec46de35cdc3aaff671592dfab987210560580

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 93a64097bb1e2a0199a7cac1475fb6ef6f62dccf — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 93a6409

Verdict parsed from the review's SHA-scoped output markers for commit 93a64097bb1e2a0199a7cac1475fb6ef6f62dccf.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 93a64097bb1e2a0199a7cac1475fb6ef6f62dccf false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 93a64097bb1e2a0199a7cac1475fb6ef6f62dccf and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 93a6409

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 93a64097bb1e2a0199a7cac1475fb6ef6f62dccf false-positive: <reason, 20 to 500 characters>

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to ship.

Verified at 328fa00:

  • I compared each of the 43 files at the merge base and at head with docstrings stripped from the AST: 0 files differ in code. Only docstring text changed.
  • I read every rewritten docstring. Each keeps the rule and the reason the test pins, now in the present tense. None was emptied, and no added line is over 100 columns.
  • No added line still carries review-round, commit or issue history.

One small nit, inline: three docstrings in test/test_file_change_snapshots.py now open with an identifier whose first letter was capitalised.

CI: Backend Tests shard 4 (Linux and Windows) fails on two tests in test/test_telegram_dispatcher.py, a file this PR does not touch. Main fixed those tests in #24531 after this branch's merge base, so a rebase onto main should turn the shard green.

Leaving this as a comment, not an approval: the change spans 43 files, above the size where a maintainer signs off by hand.

Comment thread test/test_file_change_snapshots.py Resolved
@iamwhatever
iamwhatever force-pushed the chore/test-docstrings-history branch from 328fa00 to d44b076 Compare October 26, 2026 21:09
@iamwhatever

Copy link
Copy Markdown
Contributor Author

Disposition: Backend Tests (3.12, 4) and (Windows) (4) failed on test/test_telegram_dispatcher.py::TestABusyReplyCarriesItsContext (assert [] == ['telegram_reply_parent']). That file is not touched here. The base (20c46e9) predates #24531, which fixed those tests on main. Rebased onto main a3ee04f and pushed d44b076. goal_scan.py astcheck a3ee04f574: files checked 43, code-changed 0, docstring-only 43.

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the rebase. Re-checked at d44b076.

Verified:

  • I compared each of the 43 files at the new merge base and at head with docstrings stripped from the AST: 0 files differ in code. Only docstring text changed.
  • No added line still carries review-round, commit or issue history, and no added line is over 100 columns.

Still open from the last round: the three docstrings in test/test_file_change_snapshots.py at lines 852, 926 and 944 still open with ReplaceAll, StrReplace and OldStr. The code spells these replaceAll, strReplace and oldStr. Details inline.

CI: Backend Tests (Windows) shard 6 fails in test/test_slack_team_id_env.py, a file this PR does not touch. That test arrived in the commit this branch is based on, so the failure comes from main, not from this change.

Leaving this as a comment, not an approval: the change spans 43 files, above the size where a maintainer signs off by hand.

Comment thread test/test_file_change_snapshots.py Resolved
@iamwhatever

iamwhatever commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor Author

Disposition: Backend Tests (Windows) (6) fails on six test/test_slack_team_id_env.py::TestSpawnEnv tests (FileNotFoundError: [WinError 2]). That file is not touched here; it arrived on main with #22380 (a3ee04f). Both docstring PRs fail the same six tests on the same shard, and main CI on a3ee04f has not finished that shard yet. No open fix PR found. No change on this branch.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 26, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor Author

Windows shard 6 red is main's: main CI on a3ee04f fails the same six test/test_slack_team_id_env.py::TestSpawnEnv tests with WinError 2. Tracked in #24612; open fixes #24609 and #24613. Waiting for one to merge, then rebasing. No change here.

@iamwhatever
iamwhatever force-pushed the chore/test-docstrings-history branch from d44b076 to 2a7a16e Compare October 26, 2026 23:13
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 26, 2026

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at 2a7a16e.

The earlier note is addressed: the docstrings in test/test_file_change_snapshots.py that began with a capitalised identifier now read "A replaceAll edit ...", "The edit strReplace(...) ...", and "The oldStr needle ...", so the code names keep their real casing.

What I verified on this head:

  • Parsed the base and head version of every one of the 43 changed files and compared the ASTs with docstrings blanked. All 43 are identical, so the change is docstring-only and no test logic moved.
  • No comment lines or non-Python files changed.

The one red check, Backend Tests (Windows) shard 6, fails in test/test_slack_team_id_env.py with FileNotFoundError. That file is not touched here and a docstring edit cannot change its behaviour, so a rerun should clear it.

Looks good to ship from my side. This PR changes more than 20 files, so I am leaving a comment rather than an approval; a maintainer should approve once readiness passes.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 27, 2026
Cut review-round, reviewer-found and PR/commit references from test
docstrings and comments, keeping the rule each test pins. Docstrings and
comments only; test code is unchanged.
@iamwhatever
iamwhatever force-pushed the chore/test-docstrings-history branch from 2a7a16e to 93a6409 Compare October 27, 2026 01:17
@iamwhatever

Copy link
Copy Markdown
Contributor Author

Rebased onto main 8a518e1, which carries #24609 (the fix for Windows shard 6, test/test_slack_team_id_env.py). New head 93a6409. goal_scan.py astcheck vs that merge-base: 43 files, 0 code changed.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 27, 2026
@chenmingwei23
chenmingwei23 merged commit b32db1d into main Oct 27, 2026
107 checks passed
@chenmingwei23
chenmingwei23 deleted the chore/test-docstrings-history branch October 27, 2026 06:04
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants