Repository navigation

feat(eval): NLI bake-off harness and its first result - #12

Merged

Yigtwxx merged 10 commits into

mainfrom

feat/nli-bakeoff

Oct 1, 2026

Merged

feat(eval): NLI bake-off harness and its first result#12
Yigtwxx merged 10 commits into
mainfrom
feat/nli-bakeoff

Commits on Oct 1, 2026
- docs: add the NLI bake-off design and the next-work backlog
  
  Yigtwxx
  and
  claude
  committed
  9870a02
- docs: add the NLI bake-off implementation plan
  
  Yigtwxx
  and
  claude
  committed
  354f274
- feat(eval): add the bake-off's stored inputs (snapshot, items, probabilities)
  
  Yigtwxx
  and
  claude
  committed
  15059c4
- feat(eval): add the bake-off's aggregation variants and Wilson interval
  
  Yigtwxx
  and
  claude
  committed
  52f7db8
- feat(eval): calibrate on SciFact train and score dev and AVeriTeC
  
  Yigtwxx
  and
  claude
  committed
  438ad10
- feat(eval): add the bake-off decision rule, steps in a fixed order
  
  Yigtwxx
  and
  claude
  committed
  fbbc071
- feat(eval): add the bake-off script's snapshot and score steps
  
  Yigtwxx
  and
  claude
  committed
  c61855a
- feat(eval): add the bake-off report, with coverage stated in the file
  
  Yigtwxx
  and
  claude
  committed
  6fb54fc
- feat(eval): harden the bake-off and record its first result
  
  Yigtwxx
  and
  claude
  committed
  1a74749
- docs(changelog): note the NLI bake-off under Unreleased
  
  Yigtwxx
  and
  claude
  committed
  c5c88f4