huggingface/lighteval

Pull Requests Commits

Registry updates - Infer few_shots as well as the suite

LysandreJik committed 178 days ago

719580f7

Remove the protection on "extended"

LysandreJik committed 178 days ago

be0e530f

Rework the imports to be more versatile

LysandreJik committed 178 days ago

2b88b41f

Update docs and enforces google's docstring style (#941)

NathanHB committed 179 days ago

Verified 7ed2636e

Registry refactorisation (#937)

clefourrier committed 186 days ago

Verified 1036929d

reduces cli args redundancy (#932)

NathanHB committed 190 days ago

Verified ee31223d

Fixe Sampling Metrics and Evals (#938)

NathanHB committed 191 days ago

Verified 0b288e3c

Fix `TGI` (Text Generation Inference) Endpoint Inference and TGI JSON Grammar Generation (#502)

cpcdoy committed 193 days ago

Verified da8466bd

Parametrizing the sampling evals from the CLI (#926)

clefourrier committed 193 days ago

Verified 52d3d333

Save reasoning_content from litellm as details (#929)

muupan committed 196 days ago

Verified 50756ef2

Add Filipino Benchmark Community Tasks (#852)

ljvmiranda921 committed 196 days ago

Verified d950a1e4

Test for the xet hub team (#928)

clefourrier committed 196 days ago

Verified f586400c

fix tasks list (#906)

alielfilali01 committed 196 days ago

Verified 0da128f5

Fix #878: load tasks before models (#931)

clefourrier committed 196 days ago

Verified 222a31cd

Trackio when available (#930)

NathanHB committed 198 days ago

Verified 0da24778

Upgrade to datasets 4.0 (#924)

NathanHB committed 198 days ago

Verified e53587ef

fix context size check in sglang model (#787)

xgwang committed 198 days ago

Verified 63424f4e

Update translation_literals.py - add romanian (#923)

dianaonutu committed 198 days ago

Verified bab9f6db

Reasoning tags removal is true by default (#925)

clefourrier committed 199 days ago

Verified 4e1bb325

fixes from_model function and adds tests (#921)

NathanHB committed 200 days ago

Verified 4d2bf42c

Restore `avg@4` and `avg@8` for GPQA-D (#922)

lewtun committed 203 days ago

Verified 458a3e96

Added ability to configure concurrent_requests in litellm_model.py (#911)

dameikle committed 204 days ago

Verified b0edbc34

Use lower bound for httpx (#920)

lewtun committed 205 days ago

Verified 76e5affe

making inspect working again (#887)

uralik committed 205 days ago

Verified af52de69

Trying to fix xet bug in test suite (#917)

clefourrier committed 207 days ago

Verified ecef2c66

Fix path in cache (#915)

clefourrier committed 207 days ago

Verified a88a5269

Adds the option to override the chat template (#914)

clefourrier committed 207 days ago

Verified 02bf1355

Update dep (#913)

clefourrier committed 207 days ago

Verified af211932

Caching samples PR (#909)

clefourrier committed 207 days ago

Verified df3a82d9

Fix vLLM doc (#912)

lewtun committed 207 days ago

Verified bfa60767

Older