Review Reason Fact Check

[FrankRay78]

Why

Every /speckit.reviewissue gap ends with a recommendation and a one-sentence Reason, and nothing checked that the Reason was true. The review on #270 justified a call with "Sonnet is the harness's standing default", and no file on main names Sonnet. Since #281 made reviews short enough to accept from the table, a false Reason can quietly settle a real decision.

What changes

• Before posting, the reviewer re-checks every factual claim in every Reason against the file, search or issue text it names. A claim about how the repo or harness behaves names no file, so it's checked by searching for where that behaviour would be set, and finding nothing fails it.
• A claim found false is dropped, and the Reason is rewritten as my judgement — <why> without it. A claim the run can't check may stay, but only inside a judgement Reason. The Recommendation always stays, and a failed check never blocks posting.
• The pre-post self-check, the recommendation quality bar and step 6 (the re-run rules) are aligned with this. A re-run checks only the gap it re-framed.

Non-obvious things a reviewer should know

• The judgement label never carries a false claim. The first draft only said to "recast" a failed Reason as judgement, which would have let the Add scripts/chain.sh — run the SDLC command chain end to end for one issue #270 claim through as "my judgement — Sonnet is the standing default". The /verify review caught it, and the rule now drops the claim.
• Automated reviews vs local runs. speckit-reviewissue.yml applies this prompt as written, so it's covered with no workflow change. But its tools can't read other issues (no gh issue view), so a true Reason citing another issue posts as judgement from the workflow and as evidence from a local run. Allowing Bash(gh issue view:*) would close that gap, but it widens the workflow's permissions and wasn't in Check that a /speckit.reviewissue recommendation's Reason holds up against what it cites #287's scope, so it's left for a separate decision.
• No CI gate. The check runs over a comment composed at runtime, like the rest of the self-check. It's untested in a live run, so the first review after merge is its first real test.
• "Judgement call" wording removed. The prompt no longer calls every recommendation a "judgement call", so "my judgement" marks only a Reason with no evidence behind it.

Related

Closes #287

• Make a /speckit.reviewissue comment triageable, and stop the category sweep inflating it #281 / Triageable Review #284 — made the review triageable; its open questions named this issue.
• The rationale is in CIR 2026-09-12-review-reasons-survive-checking.

🤖 Generated with Claude Code

https://claude.ai/code/session_016g3xHRn9ChW4mcjvduzB2U

[FrankRay78]

@claude Review this pull request. Analyse the code changes and provide feedback covering:

• Bugs or correctness issues
• Security concerns (including any flagged by static analysis)
• Adherence to the project conventions in CLAUDE.md
• Test coverage — do the tests adequately cover the new behaviour?
• Any spec/test-plan mismatches

End the review with a recommendation whether to merge the PR, and if not, what you suggest needs addressing first.

[claude]

Claude finished @FrankRay78's task in 3m 39s —— View job

---

Review complete

• Gather context (diff, changed files, CLAUDE.md conventions)
• Analyze .claude/commands/speckit.reviewissue.md changes for correctness
• Check CIR docs for consistency
• Check test coverage
• Post review with recommendation

Summary

This PR adds a fact-checking self-check to /speckit.reviewissue: every factual claim in a Recommendation's Reason is re-verified before posting, a false claim is dropped (Reason rewritten as my judgement — <why>), and an unverifiable claim may survive only inside a judgement Reason. It's a prompt-only change plus two CIR doc updates.

Correctness

The prompt edits are internally consistent:

• Lines 85–97 and line 183 correctly drop "judgement call" from the generic Recommendation wording (per the PR's stated "judgement call wording removed" fix), and I confirmed no other reference to that generic phrasing was left stale (git grep -i "judgement call" in this file only hits the addition's own my judgement label — good).
• The new Every Reason survives being checked paragraph correctly avoids the first-draft mistake the PR description flags (letting a false claim survive relabelled as judgement) — it explicitly says "a my judgement label never carries a claim already found false."
• The pre-post self-check bullet list (line 223) and the step-6 re-framing rule (line 291) were both updated in lockstep with the new rule, matching the PR body's claim that these three places are aligned.
• I fact-checked the PR's own factual claims (fittingly, given what this PR does): git grep -i sonnet origin/main returns no matches, confirming the Add scripts/chain.sh — run the SDLC command chain end to end for one issue #270 RED example; and .github/workflows/speckit-reviewissue.yml's --allowedTools is indeed Read,Glob,Grep,Write,Bash(gh issue comment:*),Bash(gh issue edit:*) — no gh issue view — confirming the CIR's claim about the automated-workflow gap for Reasons citing other issues. Both check out.

Minor nit: line 231 ("...or, where there is nothing checkable to cite, says it is your judgement.") and the opening sentence of the very next paragraph, Every Reason survives being checked (line 233), state the same rule back to back. Not a bug, just a candidate for tightening given this document's own emphasis on brevity elsewhere. Fix this →

Security

No security-relevant surface — this changes agent-prompt prose only, no executable code, no new shell invocations or permissions.

CLAUDE.md / constitution adherence

• Correctly invokes Principle I's config/tooling carve-out: this is a prompt change, and commit 9f793e0 records RED (git grep -i sonnet origin/main exits 1 — the false Add scripts/chain.sh — run the SDLC command chain end to end for one issue #270 claim would have failed the new check) and GREEN (rule + self-check present) evidence directly in the commit message, which I verified is accurate.
• CIR added at docs/change-intent-records/2026-09-12-review-reasons-survive-checking.md, correctly cross-linked from the existing 2026-09-12-triageable-review-comment.md's open-question bullet it resolves in part — matches this repo's CIR convention.
• No public API, CLI flag, or doc (README/USER_GUIDE) touched, so none of those update obligations apply here.

Test coverage

None expected or needed: this is a natural-language prompt for an LLM, not testable via xUnit, and per the constitution's carve-out the RED/GREEN evidence is the real check (git grep) run before/after, documented in the commit. The PR body is transparent that the self-check itself is "untested in a live run" — a reasonable, disclosed limitation given there's no committed artifact for a gate to check against (same precedent as the prior CIR).

Spec/test-plan mismatches

None — this issue has no specs/287-* folder (it's a direct prompt fix referencing #287, not a spec-kit-driven feature), so there's no spec.md/test-plan.md to reconcile against.

Environment note (not a PR defect)

The checkout this review ran in has an uncommitted local modification to .claude/commands/speckit.reviewissue.md that fully reverts it to the pre-PR (origin/main) content — working tree ≠ HEAD. This appears to be sandbox/checkout artifact unrelated to the actual PR commits (which I reviewed via git show HEAD:... and git diff origin/main...HEAD), but flagging in case it's unexpected on your end.

Recommendation

Merge. The change is small, well-scoped, self-consistent across the three places it touches, and its own RED/GREEN evidence and cross-references check out under verification. Only the minor prose-redundancy nit above is worth a follow-up touch-up, and it doesn't block merging.
· Branch: feature/287-review-reason-fact-check

[FrankRay78]

Pushed d31a742 — a one-line prompt-text trim, no behaviour change.

The Recommendation quality bar paragraph ended with "— or, where there is nothing checkable to cite, says it is your judgement", which restates the opening of Every Reason survives being checked directly below it. That paragraph states the rule in full, including the Reason: my judgement — <why>. label format, so the clause is dropped.

@claude please review the last commit (d31a742).

---

Generated by Claude Code

[claude]

Claude finished @FrankRay78's task in 1m 21s —— View job

---

Reviewing commit d31a742

• Look at the commit diff
• Check the surrounding prose for correctness/redundancy
• Post review

Summary

d31a742 trims exactly the redundant clause I flagged as a nit on the previous review: .claude/commands/speckit.reviewissue.md#L231 no longer ends with "...or, where there is nothing checkable to cite, says it is your judgement" — the Every Reason survives being checked paragraph immediately below still states that rule in full, including the Reason: my judgement — <why>. label format.

Verification

• Confirmed via git show HEAD:.claude/commands/speckit.reviewissue.md that the Recommendation quality bar paragraph now ends cleanly at "...not a restatement of the recommendation." and the following Every Reason survives being checked paragraph is untouched and still complete.
• git grep -n "nothing checkable to cite" HEAD returns no hits anywhere in the repo — the removed clause isn't echoed or relied on elsewhere (e.g. step 6's refine rule, the CIR, or the study doc).
• No other paragraph references the trimmed clause specifically; the two other places that were aligned with the fact-check rule in the prior commits (the pre-post self-check bullet list and step 6's re-framing rule) are unaffected by this trim and remain internally consistent.
• Prose reads correctly post-edit — no dangling connective or broken sentence.

Correctness / conventions

• Pure prompt-text change, no behaviour change, matches the commit message. No test or RED/GREEN evidence needed here — unlike the earlier commits in this PR (which changed the actual checking rule and cited git grep -i sonnet as RED/GREEN), this commit removes dead prose with no functional effect, so there's nothing to gate.
• Commit message is accurate and clearly scoped.

Note (carried over, not a defect of this commit)

The working tree in this checkout still has the same uncommitted local modification I flagged on the last review — it fully reverts .claude/commands/speckit.reviewissue.md to pre-PR (origin/main) content. I again reviewed via git show HEAD:... rather than the working copy, so this commit's actual committed content is what's assessed above. Flagging again in case it's unexpected in your environment.

Recommendation

No issues found in d31a742. It's a clean, minimal, well-verified trim — nothing further needed.
· Branch: feature/287-review-reason-fact-check