All pull requests

Pull requests

• Fix SciCode step keys after a context-window error

  #1545·damienen opened Nov 2, 2026·

  ·
  1

  1

• Use the last "Judgement:" label in is_correct_judgement

  #1543·damienen opened Nov 2, 2026·

  ·
  1

  1

• Reject a user-supplied ++output_file in generation extra arguments

  #1541·MohammedAlkindi opened Oct 23, 2026·

  ·
  1

  1

• Explain HF_HOME must be the container path when it is not mounted

  #1540·MohammedAlkindi opened Oct 23, 2026·

  ·
  1

  1

• feat(audio): align ASR scoring and support explicit Ray launches

  #1539·vmendelev opened Oct 15, 2026Collaborator·

  ·
  1

• fix: reset the IFEval keys where they actually live

  #1538·v0ropaev opened Oct 12, 2026·

  ·
  1

• Fix prepare_data --data_dir support without --cluster

  #1536·NathanBenDev opened Oct 3, 2026·

  ·
  1

  1

• Keep an integer answer that does not fit in float64

  #1535·SashaMIT opened Sep 30, 2026·

  ·
  3

• fix(arena): mark over-length samples as candidate losses

  #1534·rwinkelman opened Sep 28, 2026·1 task done·

  ·
  1

  2

• fix: raise a clear error on empty eval group config files

  #1532·Ashfaqbs opened Sep 16, 2026·

  ·
  1

  2

• Apply BFCL Java/JavaScript preprocessing to simple_java and simple_javascript categories

  #1528·azrabano23 opened Sep 4, 2026·

  ·
  1

  1

• Restore boxed-first default for relaxed answer extraction, add opt-in regex_first

  #1527·azrabano23 opened Sep 4, 2026·

  ·
  1

  4

• Prefer explicit "Answer: X" line over letters inside boxed expressions in MCQ eval

  #1526·azrabano23 opened Sep 4, 2026·

  ·
  1

  3

• Add GenCorrect recipe

  #1525·SeanNaren opened Sep 4, 2026Collaborator·

  ·
  4

• docs: point Papers & Releases link at docs/releases/index.md

  #1522·MohammedAlkindi opened Aug 31, 2026·

  ·
  2

• Update SHERLOC teaser figure

  #1521·tamohannes opened Aug 28, 2026Collaborator·

  ·
  

• fix: group majority@k and reward@k by answer string (#1257)

  #1519·FazeelUsmani opened Aug 25, 2026·

  ·
  1

  2

• senior-swe-bench benchmark evaluation integration

  #1515·wasiahmad opened Aug 14, 2026Collaborator·6 tasks·

  ·
  11

• swe-atlas-qna benchmark evaluation integration

  #1513·wasiahmad opened Jul 30, 2026Collaborator·

  ·
  11

• CodeRabbit Generated Unit Tests: Add unit tests for PR changes

  #1508·coderabbitai[bot] opened Jul 9, 2026Bot·

  ·
  

• fix(security): floor litellm/wandb/lxml past known CVEs; relax stale click cap

  #1507·nicgupta-nvidia opened Jul 9, 2026Collaborator·

  ·
  11

  

• High-throughput async inference: native aiohttp fast-path, scalable async loop, and request-routing keys

  #1502·gwarmstrong opened Jul 1, 2026Collaborator·

  ·
  17

• Quality: yaml.safe_load returns None for empty YAML files, propagating as None to callers expecting dict

  #1498·kumburovicbranko682-boop opened Jun 27, 2026·3 tasks done·

  ·
  1

  3

• [WIP]Add CS-FLEURS code-switched ASR benchmark with Mixed Error Rate scoring

  #1494·Slyne opened Jun 22, 2026·

  ·
  13

• Bump litellm from 1.83.14 to 1.84.0 in /core

  dependenciespython
  #1492·dependabot[bot] opened Jun 20, 2026Bot·

  ·