huggingface/lighteval

Pull Requests Commits

Merge branch 'main' into clem-add-ifbench2

clefourrier committed 340 days ago

Verified 31a53fc5

Registry refactorisation (#937)

clefourrier committed 343 days ago

Verified 1036929d

Merge branch 'main' into clem-add-ifbench

clefourrier committed 346 days ago

Verified 4f69c6ca

fixed last tests

clefourrier committed 346 days ago

fa9a4b8d

fix registry mockup

clefourrier committed 346 days ago

65c13414

reduces cli args redundancy (#932)

NathanHB committed 347 days ago

Verified ee31223d

fix tests part 1 - also removes fewshot truncation in the task name because it's no longer used anywhere in the code logically

clefourrier committed 347 days ago

70378150

Fixe Sampling Metrics and Evals (#938)

NathanHB committed 348 days ago

Verified 0b288e3c

clefourrier committed 348 days ago

fbebba7b

clefourrier committed 349 days ago

833e67dc

incredible how much code was just pulled from ifeval

clefourrier committed 349 days ago

bc882cdd

singleton + transformer sampling fix in config

clefourrier committed 349 days ago

0adf37e4

clefourrier committed 349 days ago

09e3f97b

better suite management: we don't load all optional deps all the time

clefourrier committed 349 days ago

197acb75

unrelated but these tasks were buggy

clefourrier committed 349 days ago

cfff0e90

clefourrier committed 349 days ago

0ecd7b89

Fix `TGI` (Text Generation Inference) Endpoint Inference and TGI JSON Grammar Generation (#502)

cpcdoy committed 350 days ago

Verified da8466bd

Parametrizing the sampling evals from the CLI (#926)

clefourrier committed 350 days ago

Verified 52d3d333

Save reasoning_content from litellm as details (#929)

muupan committed 353 days ago

Verified 50756ef2

Add Filipino Benchmark Community Tasks (#852)

ljvmiranda921 committed 353 days ago

Verified d950a1e4

Test for the xet hub team (#928)

clefourrier committed 353 days ago

Verified f586400c

fix tasks list (#906)

alielfilali01 committed 353 days ago

Verified 0da128f5

Fix #878: load tasks before models (#931)

clefourrier committed 353 days ago

Verified 222a31cd

Trackio when available (#930)

NathanHB committed 354 days ago

Verified 0da24778

Upgrade to datasets 4.0 (#924)

NathanHB committed 354 days ago

Verified e53587ef

fix context size check in sglang model (#787)

xgwang committed 355 days ago

Verified 63424f4e

Update translation_literals.py - add romanian (#923)

dianaonutu committed 355 days ago

Verified bab9f6db

Reasoning tags removal is true by default (#925)

clefourrier committed 356 days ago

Verified 4e1bb325

fixes from_model function and adds tests (#921)

NathanHB committed 357 days ago

Verified 4d2bf42c

Restore `avg@4` and `avg@8` for GPQA-D (#922)

lewtun committed 360 days ago

Verified 458a3e96

Older