Pull Requests huggingface/lighteval

put lower bound on typer to use literal type bug ignore-for-release

#1042 by NathanHB was merged 2025-11-06 15:09

run all hf providers with `:all` enhancement

#1039 by NathanHB was merged 2025-11-05 15:43

remove suites and make fewshot optional refacto breaking enhancement

#1038 by NathanHB was merged 2025-11-05 15:47

add openai and inspect ai lower bound ignore-for-release

#1035 by NathanHB was merged 2025-11-04 12:58

Update huggingface-cli login to use newer hf auth login documentation

#1034 by xeophon was merged 2025-11-04 12:37

Fix inspect reasoning effrot bug ignore-for-release

#1033 by NathanHB was merged 2025-11-04 12:31

adds mmlu-pro new-task

#1031 by NathanHB was merged 2025-11-04 12:28

Adds inspectai feature enhancement

#1022 by NathanHB was merged 2025-11-03 15:56

Add test file

#1021 by benbullough was closed 2025-10-18 03:35

Fix task config metric typing to accept Metric enums

#1018 by emmanuel-ferdman was merged 2025-11-20 10:43

Fix 999: always provide parameters in the metric name to allow using several combinations bug enhancement

#1017 by clefourrier was merged 2025-10-14 14:11

Move tasks to individual files feature breaking enhancement

#1016 by NathanHB was merged 2025-10-29 11:03

add translation literals for various Indic languages (Bengali, Gujarati, Punjabi, Tamil) task-update

#1015 by rpm000 was merged 2025-10-21 21:30

Fix broken link documentation

#1014 by JoelNiklaus was merged 2025-10-14 07:51

Fix nltk import failing in IFEval bug ignore-for-release

#1013 by clefourrier was merged 2025-10-14 14:10

Fix inference providers calls bug

#1012 by clefourrier was merged 2025-10-09 13:12

Tiny fix, command is failing on some OSes bug ignore-for-release

#1011 by clefourrier was merged 2025-10-07 14:46

use inspect-ai to evaluate aime25, gsm8k and IFEval

#1010 by NathanHB was closed 2025-10-14 14:17

Fix None Doc bug ignore-for-release

#1008 by lewtun was merged 2025-10-06 08:44

Fixing mixeval bug

#1006 by clefourrier was merged 2025-10-14 14:10

Revert extraction setting for `IndicesExtractionConfig` ignore-for-release

#998 by cmpatino was merged 2025-09-30 10:59

Cleanup tasks to easely remove suites

#993 by NathanHB was closed 2025-11-05 09:56

fix `lighteval task inspect` command and tiny bench task bug ignore-for-release

#992 by NathanHB was merged 2025-11-05 10:24

Fixing naming for sample evals + adding reqs in aime24 task-update science-team

#989 by clefourrier was merged 2025-10-14 14:51

Fix AvgAtK num_samples method bug ignore-for-release

#987 by amstu2 was merged 2025-09-23 11:35

Fix gsm8k HF repo bug ignore-for-release

#986 by amstu2 was merged 2025-09-23 11:13

Split up enhancement and features in release notes template ignore-for-release enhancement

#984 by NathanHB was merged 2025-10-14 14:06

Adding New Task SLR-Bench as a Community Task : Scalable Logical Reasoning Benchmark new-task

#983 by Ahmad21Omar was merged 2025-09-25 09:31

fx lcb metric bug ignore-for-release

#981 by NathanHB was merged 2025-09-23 08:28

Update vllm version constraint in pyproject.toml

#978 by clefourrier was closed 2025-09-23 08:15