[dolfim-ibm]

An audit of the AI triage and review runs after #4730 (44 runs, 34 triage comments, 26 reviews with 16 inline findings, each checked against the code) found these problems:

• 7 of 26 reviews were lost. When Bob Shell reaches its turn or cost limit, it writes an error event line before the result line. extract parsed the output as one JSON document and failed with Extra data: line 2 column 1 (char 129) (fix(pptx): move footer placeholders to the furniture layer #4746, feat(pptx,odf): keep slide video and audio and record their media file #4752 twice, fix(pptx): keep a hyperlink attached to a whole shape #4787, feat: add LightOnOCR-3 presets and grounded output parsing #4789). fix(reading-order): merge section headers that wrap across lines #4751 failed on text after the answer. On fix: flatten transparent pictures onto white before enrichment models #4776, one finding with a path outside the PR discarded the whole review.
• The triage escaped the backticks of code spans, so code names in almost every triage comment showed as literal backticks. ci: make the AI first review precise and stop review loops #4730 added code-span handling only in the review script.
• Stale ai:review-* labels. Only a push resets the labels, so a manual run or /ai triage left labels of an older commit (fix(odf): keep ODS pictures anchored to the sheet #4678, feat(ocr): detect page orientation for RapidOCR #4709, fix: fix encoding bugs in format detection (UTF-8 -> latin-1 fallback) #4726).
• Prompt issues. The triage rated a closed, unmerged PR (fix(reading-order): merge section headers that wrap across lines #4751 vs fix(reading-order): merge wrapped section headers #4155) and a 2-line overlap (fix(vlm): fix black picture crops and lost text with Granite-Docling on MLX #4642 vs fix(vlm): guard cleanup logging during shutdown #4142) as duplicates, raised CI concerns from guesses (fix(cli): correct external plugin discovery #4676, feat(email): parse email attachments #3978), and wrote internal names into comments ("Already flagged by risk.forced", "The candidate diff file is empty"). The review used blocker for an off-by-one in a cap (feat(xbrl): support XBRL ZIP and iXBRL input formats #4689) and an intended behavior (feat(ASR): add transformers and MLX presets for the ASR pipelines #4771), split one root cause into three blockers (feat(ocr): add font sanity heuristics and corrupt text OCR routing #4643), asked the reviewer to verify points the code or tests already settle, and wrote relative links that break on GitHub (feat(ocr): add font sanity heuristics and corrupt text OCR routing #4643).

Changes:

• read_bob_output reads the event lines and reports why Bob Shell stopped (::warning::Bob Shell stopped early: ...). When a limit stops the task, the last message is usually a tool result, so the review is still lost, but the warning says why instead of "Extra data". extract_answer takes the last top-level object that has the answer keys: a draft object or text after the answer no longer breaks the parse, and a nested object of a broken answer is not read as an empty review.
• A finding with a bad path, line, or severity is dropped with a warning. The other findings stay.
• The code-span helpers move from ai_pr_review.py to ai_pr_triage.py. The triage renders model text with markdown_inline: code spans stay, the rest is escaped. A pipe in a code span in a table cell is escaped, so the cell does not split.
• The triage publish step removes ai:review-* labels when the last AI review is not of the head commit. A review of the head in the same run sets its label again after this.
• Prompts:
  • triage: stricter duplicate, concerns based on a diff line, no internal names or input problems in the answer;
  • review: stricter blocker, one root cause per finding, settle checks instead of handing them to the reviewer, a turn budget, and file references as code.
• .github/CI_LABELS.md documents the duplicate rule and the review-label reset.

Checklist:

• Documentation has been updated, if necessary.
• Examples have been added, if necessary.
• Tests have been added, if necessary.

Review evidence:

• New tests:
  • a limit-stopped output in the real two-line shape reports "stopped early";
  • the last answer object wins over a draft, and a broken answer is not read from a nested object;
  • a bad finding is dropped and the others are kept;
  • code spans and a pipe in a table-cell code span render correctly;
  • stale review labels.
• A comment with code spans, a table-cell pipe, a link, a fence, and a mention, rendered with the GitHub Markdown API: code spans render as <code>, the cell stays one cell, and the link, fence, and mention stay literal text.
• pytest tests/test_ai_pr_event.py tests/test_ai_pr_triage.py tests/test_ai_pr_review.py: 38 passed. ruff, ty check .github/scripts, make validate: passed.
• Not verified: the effect of the prompt changes. This needs a few days of runs.

[github-actions]

✅ DCO Check Passed

Thanks @dolfim-ibm, all your commits are properly signed off. 🎉

[mergify]

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!