huggingface/lighteval

Pull Requests Commits

Registry updates - Infer few_shots as well as the suite

LysandreJik committed 103 days ago

719580f7

Remove the protection on "extended"

LysandreJik committed 103 days ago

be0e530f

Rework the imports to be more versatile

LysandreJik committed 103 days ago

2b88b41f

Update docs and enforces google's docstring style (#941)

NathanHB committed 105 days ago

Verified 7ed2636e

Registry refactorisation (#937)

clefourrier committed 112 days ago

Verified 1036929d

reduces cli args redundancy (#932)

NathanHB committed 116 days ago

Verified ee31223d

Fixe Sampling Metrics and Evals (#938)

NathanHB committed 117 days ago

Verified 0b288e3c

Fix `TGI` (Text Generation Inference) Endpoint Inference and TGI JSON Grammar Generation (#502)

cpcdoy committed 119 days ago

Verified da8466bd

Parametrizing the sampling evals from the CLI (#926)

clefourrier committed 119 days ago

Verified 52d3d333

Save reasoning_content from litellm as details (#929)

muupan committed 122 days ago

Verified 50756ef2

Add Filipino Benchmark Community Tasks (#852)

ljvmiranda921 committed 122 days ago

Verified d950a1e4

Test for the xet hub team (#928)

clefourrier committed 122 days ago

Verified f586400c

fix tasks list (#906)

alielfilali01 committed 122 days ago

Verified 0da128f5

Fix #878: load tasks before models (#931)

clefourrier committed 122 days ago

Verified 222a31cd

Trackio when available (#930)

NathanHB committed 124 days ago

Verified 0da24778

Upgrade to datasets 4.0 (#924)

NathanHB committed 124 days ago

Verified e53587ef

fix context size check in sglang model (#787)

xgwang committed 124 days ago

Verified 63424f4e

Update translation_literals.py - add romanian (#923)

dianaonutu committed 124 days ago

Verified bab9f6db

Reasoning tags removal is true by default (#925)

clefourrier committed 125 days ago

Verified 4e1bb325

fixes from_model function and adds tests (#921)

NathanHB committed 126 days ago

Verified 4d2bf42c

Restore `avg@4` and `avg@8` for GPQA-D (#922)

lewtun committed 129 days ago

Verified 458a3e96

Added ability to configure concurrent_requests in litellm_model.py (#911)

dameikle committed 130 days ago

Verified b0edbc34

Use lower bound for httpx (#920)

lewtun committed 131 days ago

Verified 76e5affe

making inspect working again (#887)

uralik committed 131 days ago

Verified af52de69

Trying to fix xet bug in test suite (#917)

clefourrier committed 132 days ago

Verified ecef2c66

Fix path in cache (#915)

clefourrier committed 133 days ago

Verified a88a5269

Adds the option to override the chat template (#914)

clefourrier committed 133 days ago

Verified 02bf1355

Update dep (#913)

clefourrier committed 133 days ago

Verified af211932

Caching samples PR (#909)

clefourrier committed 133 days ago

Verified df3a82d9

Fix vLLM doc (#912)

lewtun committed 133 days ago

Verified bfa60767

Older