Issues

• SciCode: after a context-window error, later steps are written under the previous step's key, so the last sub-step is never scored

  NVIDIA-NeMo/Skills#1544 · Filter by author damienen opened on Nov 2, 2026

• is_correct_judgement reads the first "Judgement:" label instead of the judge's final verdict, so an answer that contains "Judgement: Yes" is graded correct

  NVIDIA-NeMo/Skills#1542 · Filter by author damienen opened on Nov 2, 2026

• ArenaMetrics.get_incorrect_sample marks over-length generations as strong candidate wins, so --max_seq_len inflates the Arena-Hard score

  NVIDIA-NeMo/Skills#1533 · Filter by author shaurya416 opened on Sep 24, 2026

• Verify evals on Papers with Code

  NVIDIA-NeMo/Skills#1516 · Filter by author NielsRogge opened on Aug 16, 2026

• [Bug] GPQA MCQ answer extraction prefers boxed equation over final "Answer: D"

  NVIDIA-NeMo/Skills#1511 · Filter by author yingguo-trt opened on Jul 26, 2026

• BFCL v3 preprocessing differs from actual BFCL Repository, causing incorrect evaluation for Java/JavaScript splits

  NVIDIA-NeMo/Skills#1510 · Filter by author mohit-lenka-23302 opened on Jul 23, 2026

• fix: yaml.safe_load returns none for empty yaml files, propagating as none to callers expecting dict

  NVIDIA-NeMo/Skills#1497 · Filter by author kumburovicbranko682-boop opened on Jun 27, 2026

• Add IHEval (Instruction Hierarchy Evaluation) benchmark

  NVIDIA-NeMo/Skills#1463 · Filter by author bzantium opened on May 27, 2026
  3

• Add MT-Bench and Komt-Bench (Korean MT-Bench) benchmarks

  benchmarks
  NVIDIA-NeMo/Skills#1377 · Filter by author bzantium opened on Apr 17, 2026
  

• Explore options for tool call execution concurrency

  NVIDIA-NeMo/Skills#1369 · Filter by author gwarmstrong opened on Apr 16, 2026

• Merge DSBenchEvaluator into MathEvaluator

  NVIDIA-NeMo/Skills#1268 · Filter by author sgunasekar opened on Feb 22, 2026
  

• Add docs for CritPt benchmark

  NVIDIA-NeMo/Skills#1266 · Filter by author Kipok opened on Feb 21, 2026
  1

  

• Refactor registering external repositories

  NVIDIA-NeMo/Skills#1260 · Filter by author Kipok opened on Feb 19, 2026

• BaseMetrics:_compute_majority_at_k and MathMetrics:_compute_reward_at_k handling of duplicate answer strings could potentially be incorrect for non-deterministic judges

  NVIDIA-NeMo/Skills#1257 · Filter by author sgunasekar opened on Feb 19, 2026
  2

• Add a proper ci check for broken links and other issues in docs

  NVIDIA-NeMo/Skills#1207 · Filter by author Kipok opened on Feb 2, 2026

• Megatron server

  NVIDIA-NeMo/Skills#1176 · Filter by author Jorjeous opened on Jan 20, 2026
  

• is_mounted_filepath incorrect logic does not allow different paths inside and outside of container

  NVIDIA-NeMo/Skills#1173 · Filter by author esnvidia opened on Jan 19, 2026
  5

• compute_eval

  NVIDIA-NeMo/Skills#1171 · Filter by author Jorjeous opened on Jan 16, 2026
  9

• Remove patch for litellm logging

  NVIDIA-NeMo/Skills#1168 · Filter by author Kipok opened on Jan 14, 2026

• Unify answer bench and grading bench

  NVIDIA-NeMo/Skills#1163 · Filter by author Kipok opened on Jan 12, 2026

• Trend Anomalies in Ruler V2

  NVIDIA-NeMo/Skills#1143 · Filter by author zhejunliux opened on Dec 25, 2025
  5

• Redistributing input data across chunks upon job rerun or resume

  NVIDIA-NeMo/Skills#1121 · Filter by author ushnish opened on Dec 17, 2025

• Pipeline for nemotron-math dataset release

  NVIDIA-NeMo/Skills#1110 · Filter by author wedu-nvidia opened on Dec 15, 2025
  

• Default regex/boxed order changed in math grader

  bug
  NVIDIA-NeMo/Skills#1084 · Filter by author gwarmstrong opened on Dec 9, 2025

• Concurrency improvements for prover generation

  enhancement
  NVIDIA-NeMo/Skills#1083 · Filter by author gwarmstrong opened on Dec 9, 2025