Skip to content

Actions: huggingface/lighteval

Actions

Slow end to end tests

Actions

Loading...
Loading

Show workflow options

Create status badge

Loading
slow_tests.yaml
880 workflow runs
880 workflow runs

Filter by Event

⌥ + click/return to exclude

Filter by Status

⌥ + click/return to exclude

Filter by Branch

⌥ + click/return to exclude

Filter by Actor

⌥ + click/return to exclude
Pass max_tokens as an int on the litellm judge backend Slow end to end tests #1623: Pull request #1378 opened by gyanu2507
Action required gyanu2507:fix/litellm-judge-max-tokens-int
gyanu2507:fix/litellm-judge-max-tokens-int
Action required
Add unit tests for VLLMModel core methods (#724) Slow end to end tests #1622: Pull request #1377 opened by sneha4175
Action required sneha4175:add-vllm-model-unit-tests
sneha4175:add-vllm-model-unit-tests
Action required
Fix normalized multiple-choice probability underflow Slow end to end tests #1621: Pull request #1376 opened by YusefSyed
Action required YusefSyed:fix/normalized-mc-prob-underflow
YusefSyed:fix/normalized-mc-prob-underflow
Action required
fix(metrics): add actionable BERTScore baseline guidance Slow end to end tests #1620: Pull request #1373 opened by LarryHu0217
Action required LarryHu0217:codex/bertscore-baseline-guidance-995
LarryHu0217:codex/bertscore-baseline-guidance-995
Action required
fix(tasks): skip few-shot reuse warning for zero-shot tasks Slow end to end tests #1619: Pull request #1372 opened by LarryHu0217
Action required LarryHu0217:codex/avoid-zero-shot-fewshot-warning-801
LarryHu0217:codex/avoid-zero-shot-fewshot-warning-801
Action required
fix(judge): honor concurrent_requests for the OpenAI/TGI judge backend Slow end to end tests #1618: Pull request #1371 opened by mohansree14
Action required mohansree14:fix/judge-parallel-concurrent-requests
mohansree14:fix/judge-parallel-concurrent-requests
Action required
Key results and details output paths on the model revision Slow end to end tests #1617: Pull request #1370 synchronize by thisisreyy
Action required thisisreyy:add-revision-to-output-paths
thisisreyy:add-revision-to-output-paths
Action required
Key results and details output paths on the model revision Slow end to end tests #1616: Pull request #1370 opened by thisisreyy
Action required thisisreyy:add-revision-to-output-paths
thisisreyy:add-revision-to-output-paths
Action required
Pin prediction cache to resolved model revisions Slow end to end tests #1615: Pull request #1369 opened by LarryHu0217
Action required LarryHu0217:codex/cache-resolved-model-sha-1364
LarryHu0217:codex/cache-resolved-model-sha-1364
Action required
fix(cache): make multi-split document IDs unique Slow end to end tests #1614: Pull request #1368 opened by linhongyu510
Action required linhongyu510:fix/1363-multi-split-doc-ids
linhongyu510:fix/1363-multi-split-doc-ids
Action required
feat: add OrcaRouter as a first-class provider in the LiteLLM backend Slow end to end tests #1613: Pull request #1367 opened by kuswardhanietidims-svg
Action required kuswardhanietidims-svg:add-orcarouter-provider
kuswardhanietidims-svg:add-orcarouter-provider
Action required
fix: resolve SamplingMetric string normalize argument Slow end to end tests #1612: Pull request #1366 opened by Abelo9996
Action required Abelo9996:fix-samplingmetric-normalize-str
Abelo9996:fix-samplingmetric-normalize-str
Action required
fix: honor task-level num_samples and warn when it has no effect Slow end to end tests #1611: Pull request #1361 opened by Abelo9996
Action required Abelo9996:fix-num-samples-config
Abelo9996:fix-num-samples-config
Action required
ci: bump pr_style_bot to current style-bot-action reusable workflow Slow end to end tests #1610: Pull request #1360 opened by Abelo9996
Action required Abelo9996:fix-pr-style-bot-reusable-wf
Abelo9996:fix-pr-style-bot-reusable-wf
Action required
Fix LCB grader failing solutions that read sys.stdin.buffer Slow end to end tests #1609: Pull request #1257 synchronize by vineethsaivs
Action required vineethsaivs:fix/lcb-grader-stdin-buffer
vineethsaivs:fix/lcb-grader-stdin-buffer
Action required
Show evaluated sample counts in Markdown results Slow end to end tests #1608: Pull request #1359 opened by Labeeb2339
Action required Labeeb2339:codex/add-markdown-eval-count
Labeeb2339:codex/add-markdown-eval-count
Action required
Fix Dyck scoring to preserve bracket order Slow end to end tests #1607: Pull request #1358 opened by Excelius-Wang
Action required Excelius-Wang:fix/dyck-strict-scoring
Excelius-Wang:fix/dyck-strict-scoring
Action required
Fix continuation tokens after distributed gathering Slow end to end tests #1606: Pull request #1357 opened by Excelius-Wang
Action required Excelius-Wang:fix/logprob-token-normalization
Excelius-Wang:fix/logprob-token-normalization
Action required
Fix DROP exact match: the length check compares gold to gold Slow end to end tests #1605: Pull request #1356 opened by Ag3497120
Action required Ag3497120:fix/drop-em-multiset-check
Ag3497120:fix/drop-em-multiset-check
Action required
fix(sglang): size batch truncation from the longest input in the batch, not the first Slow end to end tests #1604: Pull request #1355 opened by shoemoney
Action required shoemoney:fix-sglang-context-size-longest-input
shoemoney:fix-sglang-context-size-longest-input
Action required
don't score an empty F1 gold reference as zero Slow end to end tests #1603: Pull request #1354 opened by gyanu2507
Action required gyanu2507:fix/f1-empty-gold
gyanu2507:fix/f1-empty-gold
Action required
docs: fix broken code examples in evaluating-a-custom-model.mdx Slow end to end tests #1602: Pull request #1352 opened by mohansree14
Action required mohansree14:fix/custom-model-docs-syntax
mohansree14:fix/custom-model-docs-syntax
Action required
Fix duplicate custom task module imports Slow end to end tests #1601: Pull request #1351 opened by yangweigbh
Action required yangweigbh:fix/reuse-custom-task-module
yangweigbh:fix/reuse-custom-task-module
Action required
fix: keep every task's responses when reloading them from details Slow end to end tests #1600: Pull request #1350 opened by tonycoder-hub
Action required tonycoder-hub:cursor/fix-details-reload-multi-task-responses-3037
tonycoder-hub:cursor/fix-details-reload-multi-task-responses-3037
Action required
Fix relational comparisons with different condition counts Slow end to end tests #1599: Pull request #1349 opened by Excelius-Wang
Action required Excelius-Wang:fix/relational-condition-count
Excelius-Wang:fix/relational-condition-count
Action required