huggingface/lighteval

Pull Requests Commits

Merge branch 'main' into clem-add-ifbench2

clefourrier committed 153 days ago

Verified 31a53fc5

Registry refactorisation (#937)

clefourrier committed 156 days ago

Verified 1036929d

Merge branch 'main' into clem-add-ifbench

clefourrier committed 159 days ago

Verified 4f69c6ca

fixed last tests

clefourrier committed 159 days ago

fa9a4b8d

fix registry mockup

clefourrier committed 159 days ago

65c13414

reduces cli args redundancy (#932)

NathanHB committed 160 days ago

Verified ee31223d

fix tests part 1 - also removes fewshot truncation in the task name because it's no longer used anywhere in the code logically

clefourrier committed 160 days ago

70378150

Fixe Sampling Metrics and Evals (#938)

NathanHB committed 161 days ago

Verified 0b288e3c

clefourrier committed 162 days ago

fbebba7b

clefourrier committed 162 days ago

833e67dc

incredible how much code was just pulled from ifeval

clefourrier committed 162 days ago

bc882cdd

singleton + transformer sampling fix in config

clefourrier committed 162 days ago

0adf37e4

clefourrier committed 162 days ago

09e3f97b

better suite management: we don't load all optional deps all the time

clefourrier committed 162 days ago

197acb75

unrelated but these tasks were buggy

clefourrier committed 162 days ago

cfff0e90

clefourrier committed 162 days ago

0ecd7b89

Fix `TGI` (Text Generation Inference) Endpoint Inference and TGI JSON Grammar Generation (#502)

cpcdoy committed 163 days ago

Verified da8466bd

Parametrizing the sampling evals from the CLI (#926)

clefourrier committed 163 days ago

Verified 52d3d333

Save reasoning_content from litellm as details (#929)

muupan committed 166 days ago

Verified 50756ef2

Add Filipino Benchmark Community Tasks (#852)

ljvmiranda921 committed 166 days ago

Verified d950a1e4

Test for the xet hub team (#928)

clefourrier committed 166 days ago

Verified f586400c

fix tasks list (#906)

alielfilali01 committed 166 days ago

Verified 0da128f5

Fix #878: load tasks before models (#931)

clefourrier committed 166 days ago

Verified 222a31cd

Trackio when available (#930)

NathanHB committed 167 days ago

Verified 0da24778

Upgrade to datasets 4.0 (#924)

NathanHB committed 168 days ago

Verified e53587ef

fix context size check in sglang model (#787)

xgwang committed 168 days ago

Verified 63424f4e

Update translation_literals.py - add romanian (#923)

dianaonutu committed 168 days ago

Verified bab9f6db

Reasoning tags removal is true by default (#925)

clefourrier committed 169 days ago

Verified 4e1bb325

fixes from_model function and adds tests (#921)

NathanHB committed 170 days ago

Verified 4d2bf42c

Restore `avg@4` and `avg@8` for GPQA-D (#922)

lewtun committed 173 days ago

Verified 458a3e96

Older