Skip to content

Complete the thirty-question RAG evaluation baseline - #8

Merged
shipitdev merged 1 commit into
mainfrom
codex/complete-thirty-question-evaluation
Nov 10, 2026
Merged

shipitdev merged 1 commit into
mainfrom
codex/complete-thirty-question-evaluation

Conversation

@shipitdev

Copy link
Copy Markdown
Owner

Changes

  • Finish the fixed 30-question training baseline and retain all failed outcomes.
  • Expand context recall to 30 cases, faithfulness to ten and precision to four valid scores.
  • Record quota-limited judge expansion, sample sizes, archived attempts and actual usage.
  • Preserve original inference code and reserved development questions.

Checks

  • Evaluation and publication tests passed with 19 passed and two optional skips.
  • Report assertions cover cohort size, outcome totals, usage, redaction and unchanged inference file hashes.
  • Saved outcomes published to LangSmith without further model calls.

@shipitdev
shipitdev merged commit e2d008d into main Nov 10, 2026
2 checks passed
@shipitdev
shipitdev deleted the codex/complete-thirty-question-evaluation branch November 10, 2026 18:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant