Pull Requests huggingface/lighteval

Remove suites in task configs example and fix task with hf_filters bug

#1051 by NathanHB was merged 2025-11-12 13:16

not forcing use_logits at True bug

#1050 by f14-bertolotti was merged 2025-11-12 10:29

wrong attribute self.k -> self.n bug

#1049 by f14-bertolotti was merged 2025-11-12 10:29

wrong attribute self.k -> self.n

#1048 by francesco-bertolotti was closed 2025-11-11 08:17

remove suites from serbian_eval.py bug

#1044 by S-Y-A-N was merged 2025-11-12 10:05

neater bundle and logdir for inspect-ai enhancement

#1043 by NathanHB was merged 2025-11-12 10:28

put lower bound on typer to use literal type bug ignore-for-release

#1042 by NathanHB was merged 2025-11-06 15:09

run all hf providers with `:all` enhancement

#1039 by NathanHB was merged 2025-11-05 15:43

remove suites and make fewshot optional refacto breaking enhancement

#1038 by NathanHB was merged 2025-11-05 15:47

add openai and inspect ai lower bound ignore-for-release

#1035 by NathanHB was merged 2025-11-04 12:58

Update huggingface-cli login to use newer hf auth login documentation

#1034 by xeophon was merged 2025-11-04 12:37

Fix inspect reasoning effrot bug ignore-for-release

#1033 by NathanHB was merged 2025-11-04 12:31

adds mmlu-pro new-task

#1031 by NathanHB was merged 2025-11-04 12:28

Adds inspectai feature enhancement

#1022 by NathanHB was merged 2025-11-03 15:56

Add test file

#1021 by benbullough was closed 2025-10-18 03:35

Fix task config metric typing to accept Metric enums

#1018 by emmanuel-ferdman was merged 2025-11-20 10:43

Fix 999: always provide parameters in the metric name to allow using several combinations bug enhancement

#1017 by clefourrier was merged 2025-10-14 14:11

Move tasks to individual files feature breaking enhancement

#1016 by NathanHB was merged 2025-10-29 11:03

add translation literals for various Indic languages (Bengali, Gujarati, Punjabi, Tamil) task-update

#1015 by rpm000 was merged 2025-10-21 21:30

Fix broken link documentation

#1014 by JoelNiklaus was merged 2025-10-14 07:51

Fix nltk import failing in IFEval bug ignore-for-release

#1013 by clefourrier was merged 2025-10-14 14:10

Fix inference providers calls bug

#1012 by clefourrier was merged 2025-10-09 13:12

Tiny fix, command is failing on some OSes bug ignore-for-release

#1011 by clefourrier was merged 2025-10-07 14:46

use inspect-ai to evaluate aime25, gsm8k and IFEval

#1010 by NathanHB was closed 2025-10-14 14:17

Fix None Doc bug ignore-for-release

#1008 by lewtun was merged 2025-10-06 08:44

Fixing mixeval bug

#1006 by clefourrier was merged 2025-10-14 14:10

Revert extraction setting for `IndicesExtractionConfig` ignore-for-release

#998 by cmpatino was merged 2025-09-30 10:59

Cleanup tasks to easely remove suites

#993 by NathanHB was closed 2025-11-05 09:56

fix `lighteval task inspect` command and tiny bench task bug ignore-for-release

#992 by NathanHB was merged 2025-11-05 10:24

Fixing naming for sample evals + adding reqs in aime24 task-update science-team

#989 by clefourrier was merged 2025-10-14 14:51