Skip to content

Add finding previews and calibration feedback tools - #4

Merged
mrubens merged 2 commits into
mainfrom
codex/feedback-tools
Sep 30, 2026
Merged

mrubens merged 2 commits into
mainfrom
codex/feedback-tools

Conversation

@mrubens

@mrubens mrubens commented Sep 29, 2026

Copy link
Copy Markdown
Contributor

Summary

Judgement findings now show the rule cutoff, observed probability, and a bounded preview of changed lines. Slow checks emit progress on stderr, while JSON output stays clean and the existing judgment cache continues to reuse matching packets.

Adds two commands to help improve rules from real feedback:

  • judgement capture previews a labeled example from staged or committed Git content, including unchanged supporting context. Saving is explicit and refuses to overwrite files.
  • judgement compare compares saved calibration reports on identical fixtures, showing regressions per example even when overall accuracy improves.

Exports a shared runCli for applications with their own inference adapter. Documents strict CI checks, fixture review, comparison, cache safety, and the distinction between changed-line previews and model localization. Prepares version 0.4.0; this PR does not publish it.

Validation

  • npm run check — 60 tests passed.
  • npm pack --dry-run --json — passed.
  • Disposable Git repository tests cover staged/unstaged separation, additions/deletions, supporting context, binary/symlink rejection, cache reuse, progress, comparison provenance, and CLI exit codes.
  • No live model calls or model prompt changes.

@mrubens
mrubens merged commit 1b2853f into main Sep 30, 2026
8 checks passed
@mrubens
mrubens deleted the codex/feedback-tools branch September 30, 2026 15:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant