Repository navigation

feat(evals): judge open-ended answers and record judge identity - #8574

Closed

sudoKrishna wants to merge 24 commits into

simstudioai:mainfrom

sudoKrishna:feat/evals-judged-scenarios

Commits on Sep 29, 2026
Commits on Sep 30, 2026
- feat(evals): compare agent tool-use across models
  sudoKrishna
  committed
  26937ff
Commits on Oct 1, 2026
Commits on Oct 2, 2026
- test(evals): address review findings on the eval harness
  sudoKrishna
  committed
  f619479
- feat(evals): judge open-ended agent answers
  sudoKrishna
  committed
  3d8de59
- feat(evals): record judge identity and refuse cross-identity comparisons
  sudoKrishna
  committed
  74faa92
- feat(evals): judge the remaining open-ended scenarios
  sudoKrishna
  committed
  2023d07
- fix(evals): address the judge review findings
  sudoKrishna
  committed
  8bf9c01
Commits on Oct 4, 2026
- feat(evals): record and replay judge transcripts
  sudoKrishna
  committed
  69af5d7