Go
Home
Pricing
FAQ
Install
Home
Pricing
FAQ
Install
Login
via GitHub
huggingface/lighteval
Pull Requests
Commits
Open
Closed
put lower bound on typer to use literal type
bug
ignore-for-release
#1042 by
NathanHB
was merged 2025-11-06 15:09
run all hf providers with `:all`
enhancement
#1039 by
NathanHB
was merged 2025-11-05 15:43
remove suites and make fewshot optional
refacto
breaking
enhancement
#1038 by
NathanHB
was merged 2025-11-05 15:47
add openai and inspect ai lower bound
ignore-for-release
#1035 by
NathanHB
was merged 2025-11-04 12:58
Update huggingface-cli login to use newer hf auth login
documentation
#1034 by
xeophon
was merged 2025-11-04 12:37
Fix inspect reasoning effrot
bug
ignore-for-release
#1033 by
NathanHB
was merged 2025-11-04 12:31
adds mmlu-pro
new-task
#1031 by
NathanHB
was merged 2025-11-04 12:28
Adds inspectai
feature
enhancement
#1022 by
NathanHB
was merged 2025-11-03 15:56
Add test file
#1021 by
benbullough
was closed 2025-10-18 03:35
Fix task config metric typing to accept Metric enums
#1018 by
emmanuel-ferdman
was merged 2025-11-20 10:43
Fix 999: always provide parameters in the metric name to allow using several combinations
bug
enhancement
#1017 by
clefourrier
was merged 2025-10-14 14:11
Move tasks to individual files
feature
breaking
enhancement
#1016 by
NathanHB
was merged 2025-10-29 11:03
add translation literals for various Indic languages (Bengali, Gujarati, Punjabi, Tamil)
task-update
#1015 by
rpm000
was merged 2025-10-21 21:30
Fix broken link
documentation
#1014 by
JoelNiklaus
was merged 2025-10-14 07:51
Fix nltk import failing in IFEval
bug
ignore-for-release
#1013 by
clefourrier
was merged 2025-10-14 14:10
Fix inference providers calls
bug
#1012 by
clefourrier
was merged 2025-10-09 13:12
Tiny fix, command is failing on some OSes
bug
ignore-for-release
#1011 by
clefourrier
was merged 2025-10-07 14:46
use inspect-ai to evaluate aime25, gsm8k and IFEval
#1010 by
NathanHB
was closed 2025-10-14 14:17
Fix None Doc
bug
ignore-for-release
#1008 by
lewtun
was merged 2025-10-06 08:44
Fixing mixeval
bug
#1006 by
clefourrier
was merged 2025-10-14 14:10
Revert extraction setting for `IndicesExtractionConfig`
ignore-for-release
#998 by
cmpatino
was merged 2025-09-30 10:59
Cleanup tasks to easely remove suites
#993 by
NathanHB
was closed 2025-11-05 09:56
fix `lighteval task inspect` command and tiny bench task
bug
ignore-for-release
#992 by
NathanHB
was merged 2025-11-05 10:24
Fixing naming for sample evals + adding reqs in aime24
task-update
science-team
#989 by
clefourrier
was merged 2025-10-14 14:51
Fix AvgAtK num_samples method
bug
ignore-for-release
#987 by
amstu2
was merged 2025-09-23 11:35
Fix gsm8k HF repo
bug
ignore-for-release
#986 by
amstu2
was merged 2025-09-23 11:13
Split up enhancement and features in release notes template
ignore-for-release
enhancement
#984 by
NathanHB
was merged 2025-10-14 14:06
Adding New Task SLR-Bench as a Community Task : Scalable Logical Reasoning Benchmark
new-task
#983 by
Ahmad21Omar
was merged 2025-09-25 09:31
fx lcb metric
bug
ignore-for-release
#981 by
NathanHB
was merged 2025-09-23 08:28
Update vllm version constraint in pyproject.toml
#978 by
clefourrier
was closed 2025-09-23 08:15
Newer
Older