Skip to content

Preserve verifier errors during output normalization - #1

Open
RainieLLM wants to merge 1 commit into
MatrAIx-ai:mainfrom
RainieLLM:fix/preserve-verifier-errors
Open

RainieLLM wants to merge 1 commit into
MatrAIx-ai:mainfrom
RainieLLM:fix/preserve-verifier-errors

Conversation

@RainieLLM

Copy link
Copy Markdown

Verifier errors lost their error fields during output normalization, so some runs were counted as model failures and skipped by --retry-errors. Keep the error metadata at the top level and add regression tests.

Checked: 886 unit tests, both verifier smoke suites, all 2,460 task checks, leaderboard recomputation, and the task manifest.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant