Skip to content

Actions: huggingface/lighteval

Actions

All workflows

Actions

Loading...
Loading

Showing runs from all workflows
2,500+ workflow runs
2,500+ workflow runs

Filter by Workflow

⌥ + click/return to exclude

Filter by Event

⌥ + click/return to exclude

Filter by Status

⌥ + click/return to exclude

Filter by Branch

⌥ + click/return to exclude

Filter by Actor

⌥ + click/return to exclude
[Proposal] OpenEval Import/Export Support .github/workflows/pr_style_bot.yaml #302: Issue comment #1318 (comment) by adhabnr-ux
Failure
Failure
fix(metrics): avg@n scores raw text instead of post-processed text, and skips preprocessing Build PR Documentation #2088: Pull request #1335 opened by arthi-arumugam-git
Action required arthi-arumugam-git:fix-avg-at-n-post-processed-text
arthi-arumugam-git:fix-avg-at-n-post-processed-text
Action required
fix(metrics): avg@n scores raw text instead of post-processed text, and skips preprocessing Tests #3986: Pull request #1335 opened by arthi-arumugam-git
Action required arthi-arumugam-git:fix-avg-at-n-post-processed-text
arthi-arumugam-git:fix-avg-at-n-post-processed-text
Action required
fix(metrics): avg@n scores raw text instead of post-processed text, and skips preprocessing Quality #3987: Pull request #1335 opened by arthi-arumugam-git
Action required arthi-arumugam-git:fix-avg-at-n-post-processed-text
arthi-arumugam-git:fix-avg-at-n-post-processed-text
Action required
fix(metrics): avg@n scores raw text instead of post-processed text, and skips preprocessing Slow end to end tests #1586: Pull request #1335 opened by arthi-arumugam-git
Action required arthi-arumugam-git:fix-avg-at-n-post-processed-text
arthi-arumugam-git:fix-avg-at-n-post-processed-text
Action required
Balanced few-shot sampling truncates on a falsy class label and uses the global random module .github/workflows/pr_style_bot.yaml #301: Issue comment #1309 (comment) by atirna
Failure
Failure
Sequential few-shot selection mutates the task's shared few-shot pool in place, corrupting later variance seeds .github/workflows/pr_style_bot.yaml #300: Issue comment #1307 (comment) by atirna
Failure
Failure
Fix few-shot sampler isolation Build PR Documentation #2087: Pull request #1334 opened by atirna
Action required atirna:fix/fewshot-sampler-isolation
atirna:fix/fewshot-sampler-isolation
Action required
Fix few-shot sampler isolation Quality #3986: Pull request #1334 opened by atirna
Action required atirna:fix/fewshot-sampler-isolation
atirna:fix/fewshot-sampler-isolation
Action required
Fix few-shot sampler isolation Slow end to end tests #1585: Pull request #1334 opened by atirna
Action required atirna:fix/fewshot-sampler-isolation
atirna:fix/fewshot-sampler-isolation
Action required
Fix few-shot sampler isolation Tests #3985: Pull request #1334 opened by atirna
Action required atirna:fix/fewshot-sampler-isolation
atirna:fix/fewshot-sampler-isolation
Action required
Fix BBEH generative evaluation Build PR Documentation #2086: Pull request #1333 opened by lakshayxi
Action required lakshayxi:fix-bbeh-evaluation
lakshayxi:fix-bbeh-evaluation
Action required
Fix BBEH generative evaluation Tests #3984: Pull request #1333 opened by lakshayxi
Action required lakshayxi:fix-bbeh-evaluation
lakshayxi:fix-bbeh-evaluation
Action required
Fix BBEH generative evaluation Slow end to end tests #1584: Pull request #1333 opened by lakshayxi
Action required lakshayxi:fix-bbeh-evaluation
lakshayxi:fix-bbeh-evaluation
Action required
Fix BBEH generative evaluation Quality #3985: Pull request #1333 opened by lakshayxi
Action required lakshayxi:fix-bbeh-evaluation
lakshayxi:fix-bbeh-evaluation
Action required
Lighteval TriviaQA evaluation not giving results as per smolLM-360M reported numbers .github/workflows/pr_style_bot.yaml #299: Issue comment #1316 (comment) by seva9523
Failure
Failure
Fix DetailsLogger compatibility with xxhash 4.0 Build PR Documentation #2085: Pull request #1332 opened by Honorllq
Action required Honorllq:agent/fix-xxhash-compat
Honorllq:agent/fix-xxhash-compat
Action required
Fix DetailsLogger compatibility with xxhash 4.0 Slow end to end tests #1583: Pull request #1332 opened by Honorllq
Action required Honorllq:agent/fix-xxhash-compat
Honorllq:agent/fix-xxhash-compat
Action required
Fix DetailsLogger compatibility with xxhash 4.0 Tests #3983: Pull request #1332 opened by Honorllq
Action required Honorllq:agent/fix-xxhash-compat
Honorllq:agent/fix-xxhash-compat
Action required
Fix DetailsLogger compatibility with xxhash 4.0 Quality #3984: Pull request #1332 opened by Honorllq
Action required Honorllq:agent/fix-xxhash-compat
Honorllq:agent/fix-xxhash-compat
Action required
[BUG] Incompatibility with latest version of xxhash .github/workflows/pr_style_bot.yaml #298: Issue comment #1330 (comment) by Honorllq
Failure
Failure
Fix inspect-ai GenerateConfig field names in lighteval eval Build PR Documentation #2084: Pull request #1331 opened by harshasiddartha
Action required harshasiddartha:fix/inspect-ai-generateconfig-field-names
harshasiddartha:fix/inspect-ai-generateconfig-field-names
Action required
Fix inspect-ai GenerateConfig field names in lighteval eval Slow end to end tests #1582: Pull request #1331 opened by harshasiddartha
Action required harshasiddartha:fix/inspect-ai-generateconfig-field-names
harshasiddartha:fix/inspect-ai-generateconfig-field-names
Action required
Fix inspect-ai GenerateConfig field names in lighteval eval Quality #3983: Pull request #1331 opened by harshasiddartha
Action required harshasiddartha:fix/inspect-ai-generateconfig-field-names
harshasiddartha:fix/inspect-ai-generateconfig-field-names
Action required
Fix inspect-ai GenerateConfig field names in lighteval eval Tests #3982: Pull request #1331 opened by harshasiddartha
Action required harshasiddartha:fix/inspect-ai-generateconfig-field-names
harshasiddartha:fix/inspect-ai-generateconfig-field-names
Action required