[getappz]

Experimental — kept in draft. The full refinement loop has not demonstrated a final accuracy benefit and is not proposed for merge yet.

The independently useful fixes are merged into master:

• fix(consensus): anonymize final judge candidates #921: anonymous consensus judging.
• fix(agents): retain terminal Claude replies before cleanup #922: preserve completed Claude replies and usage during process cleanup, including the parsing and fixture review followups.

This draft now targets master. Its diff retains the experimental loop, trajectory/evidence handling and supporting refinement infrastructure. Broader evaluation is still needed before considering it for merge.

Adds an opt-in Rust Harness2 refinement loop that proposes instruction edits, probes them against frozen task inputs, composes revisions, selects among successful baselines and finals, and retains repository/domain-scoped lessons.

Reuses the workflow engine, SQLite recovery and leases, headless agent runner, process cancellation, Git snapshots, and memory. Supports parallel/sequential modes, durable call/time budgets and receipts, validated instruction/skill paths, resume identity/artifact checks, and start/status/resume/cancel commands. Adopted harnesses are exported for inspection. Usage: src/refinement/README.md.

Feedback handling now preserves direct solver-visible observations before failure/timeout classification, delivers long observations as ordered artifact parts, and grounds improvement prompts in tool evidence. Private acceptance checks and verdicts stay outside improvement prompts. Exhausted probe retries retain the last failed receipt and its evidence instead of substituting an empty receipt; failed trials remain ineligible. Resume identity was bumped for the changed method.

Validation

• Latest focused workflow suite: 21 passed, including a regression observed failing before the exhausted-retry fix, then passing in both parallel and sequential modes.
• Scoped Clippy, formatting, whitespace checks, and independent review passed. Full Linux/Windows workspace CI runs on this PR.
• Prior workflow/consensus suites: 203 passed; native refinement integration: 22 passed; Git snapshot/shell regressions: 16 passed; headless runner suite: 53 passed, 3 subprocess fixtures ignored.

Live experiment

A complete run on a frozen Rust Git-ignore task scored baseline 679/684, unchanged-harness repeat 682/684, domain revision 684/684 on retry, and composed final 679/684. The final repeated the baseline's same five escaped-separator/globstar failures. No harness was adopted: eligible baseline/final trials failed. The intermediate all-pass result did not replicate in composition; the unchanged-harness gain also demonstrates sampling variation. This run does not establish an accuracy benefit.

The loop took 33m13s, reported $0.591 in usage, and conservatively accounted $0.864 including a timed-out call reservation. $1.814 remains within the user's current $5 allowance after earlier work. All solver traces were attached and frozen acceptance hashes were verified unchanged. The exhausted-retry evidence fix was made after this run and has not had another paid evaluation.

Earlier dogfooding found passing-task ties and a terminal-reply timeout classification bug, now fixed. No full workspace suite was run locally. Dollar caps currently require Claude Code for both roles; other agents use call/time limits. Existing host sandbox policy remains in force. Costs are reported usage plus conservative reservations, not an invoice.

---

Opened by codex on flared:c997d745ae66 for item #751 via agentflare ownership tracking. The regular codex branch was published through the GitHub API because agentflare pr open requires a managed task worktree.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}