Skip to content

[Fix] Correct MATH JudgeLLM config imports in example and guides - #2662

Open
YueshenZ wants to merge 1 commit into
open-compass:mainfrom
YueshenZ:fix/math-llm-judge-config-import
Open

YueshenZ wants to merge 1 commit into
open-compass:mainfrom
YueshenZ:fix/math-llm-judge-config-import

Conversation

@YueshenZ

@YueshenZ YueshenZ commented Oct 2, 2026

Copy link
Copy Markdown

Motivation

The MATH JudgeLLM example and both language versions of the objective JudgeLLM guide import math_llm_judge, but that config module does not exist. Loading the example with mmengine.Config.fromfile fails with ConfigParsingError: .../math_llm_judge.py not found! before evaluation can start.

Modification

Point all three imports to the existing math_llm_judge_gen config, which is also the entry used by dataset-index.yml.

Validation

  • Reproduced the missing-config error on the unchanged example and both complete documentation snippets with MMEngine 0.10.5.
  • Loaded all three complete configs after the fix and checked the resulting MATH dataset, evaluated model, judge model, and LMEvaluator settings. Documentation snippets were loaded from the config directory so their relative imports resolve as intended.
  • All applicable repository pre-commit hooks passed on the three changed files, including the package-wide dataset suffix check. On macOS/Python 3.13, this required the base interpreter and PYTHON_FROZEN_MODULES=off so MMEngine recognizes standard-library imports.
  • git diff --check passed. Model inference was not run; the change is limited to config imports.

Checklist

  • Pre-commit or other linting tools are used to fix potential lint issues.
  • The documentation has been modified accordingly.
  • CLA has been signed and all committers have signed the CLA in this PR.

AI assistance: prepared and validated with Codex.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants