Skip to content

fix(refinement): timed-out acceptance check is unknown; cap session telemetry evidence - #929

Merged
getappz merged 1 commit into
codex/harness2-refinementfrom
fix/harness2-review-fixes
Oct 20, 2026
Merged

getappz merged 1 commit into
codex/harness2-refinementfrom
fix/harness2-review-fixes

Conversation

@getappz

@getappz getappz commented Oct 19, 2026 •

Copy link
Copy Markdown
Owner

Follow-up to agentflare item #751 (NOT GitHub PR #751), from a review against docs/audits/2026-10-18-harness2-integration.md.

Changes

  • A timed-out acceptance check now yields accepted = None (unknown) instead of Some(false). The audit requires that a failed evaluator never becomes a zero score; previously a slow or flaky check could eliminate a good candidate or make the baseline look bad. Logic is in check_verdict, covered by timed_out_check_is_unknown_not_failed.
  • Session telemetry attached to receipt evidence is capped at 64K chars (tail kept, truncation noted), so long sessions no longer fan out into hundreds of evidence files for the proposer.

Verification

  • cargo test --bin agentflare refinement:: : 27 passed (plain cargo, as in the branch ledger; mbx not used)
  • cargo fmt --check clean
  • Not run: full workspace, Linux

Not addressed (follow-ups)

Summary by CodeRabbit

  • Bug Fixes
    • Timed-out acceptance checks are now reported as unknown rather than failed. Successful and non-timeout failed checks retain their results, and processing stops at the first check that does not succeed.
  • Improvements
    • Solver session telemetry is limited to the most recent 64 KiB of text. When earlier content is omitted, the telemetry includes a notice that it was truncated.

…elemetry evidence

Agentflare-Agent: claude-code_2-1-303_agent
Agentflare-Branch: fix/harness2-review-fixes
Agentflare-Session: b39fb7c8-e549-4da4-9407-4c3a3392da34
@getappz

getappz commented Oct 19, 2026

Copy link
Copy Markdown
Owner Author

Tracking this PR for automated review (flared:c997d745ae66).

@coderabbitai

coderabbitai Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: getappz/agentflare/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Essentials
  • Run ID: 68d65203-d727-47ad-b0ec-232620a4fb3c
📥 Commits

Reviewing files that changed from the base of the PR and between 03f70f4 and cd0b18e.

📒 Files selected for processing: 2
  • src/refinement/backend.rs
  • src/refinement/tests.rs

Included review availability: This review used your included allowance. 3 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 4 reviews per hour.

📝 Walkthrough

Acceptance-check processing now treats timed-out checks as unknown and stops at the first non-success verdict. Session telemetry now includes at most the final 64 KiB of characters, with a truncation notice when needed.

Changes: Refinement backend updates

Layer / File(s) Summary
Acceptance-check verdicts
src/refinement/backend.rs, src/refinement/tests.rs
check_verdict returns None for timed-out checks and Some(success) otherwise. Processing stops at the first non-success verdict. Tests cover successful, failed, and timed-out checks.
Session telemetry cap
src/refinement/backend.rs
Session telemetry includes only the final 64 KiB of characters. Evidence includes a truncation notice when the original text exceeds the cap.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to cd0b1

The timeout and telemetry changes match their stated behavior. No concrete merge-blocking regression is established in the reviewed changes.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 57.14% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 2 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Title check Passed The title clearly identifies both primary changes: treating timed-out acceptance checks as unknown and capping session telemetry evidence.
Description check Passed The description explains the purpose, implementation, verification results, known test limitations, risks, and follow-up items. It uses equivalent sections instead of the exact template headings and d…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Description check

Explanation

The description explains the purpose, implementation, verification results, known test limitations, risks, and follow-up items. It uses equivalent sections instead of the exact template headings and does not report a clippy run, but it is mostly complete.

  • Fix all pre-merge checks with AI
✨ Finishing Touches: 💡 1
📝 Generate docstrings: 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@getappz

getappz commented Oct 19, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@getappz
getappz merged commit e58942e into codex/harness2-refinement Oct 20, 2026
16 checks passed
@getappz
getappz deleted the fix/harness2-review-fixes branch October 20, 2026 02:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant