feat(evals): add LLM-as-judge scoring - #8515

Closed

sudoKrishna wants to merge 7 commits into

simstudioai:mainfrom

sudoKrishna:feat/evals-llm-judge

Commits on Sep 29, 2026

feat(evals): add agent tool-use evaluation harness
sudoKrishna
committed
86c79d7
feat(evals): add live DeepSeek model runs to the agent eval suite
sudoKrishna
committed
af2e6ca
test(evals): assert grounded behavior instead of exact phrasing in live mode
sudoKrishna
committed
433fd0f
feat(evals): run agent scenarios through the DAGExecutor
sudoKrishna
committed
85525a4
test(evals): assert executor block retry on agent failure
sudoKrishna
committed
fb7f54c
test(evals): assert executor model fallback on primary failure
sudoKrishna
committed
c93426b

Commits on Oct 1, 2026

feat(evals): add LLM-as-judge scoring
sudoKrishna
committed
4590af8