huggingface/lighteval

Pull Requests Commits

Merge branch 'main' into nathan-better-readme

NathanHB committed 161 days ago

Verified 42eff7d7

set upper bound on vllm version (#964)

NathanHB committed 161 days ago

Verified fc50ca09

NathanHB committed 161 days ago

f7a5ebb0

NathanHB committed 161 days ago

94c86673

NathanHB committed 161 days ago

bb3b8e97

NathanHB committed 161 days ago

6ff9246b

adds tasks list to readme, shortens intro paragrpah and add example for python API

NathanHB committed 161 days ago

41666bf2

MMLU Redux and Fixing the caching (#883)

clefourrier committed 165 days ago

Verified eda7eed3

Update docs and enforces google's docstring style (#941)

NathanHB committed 168 days ago

Verified 7ed2636e

Registry refactorisation (#937)

clefourrier committed 175 days ago

Verified 1036929d

reduces cli args redundancy (#932)

NathanHB committed 179 days ago

Verified ee31223d

Fixe Sampling Metrics and Evals (#938)

NathanHB committed 180 days ago

Verified 0b288e3c

Fix `TGI` (Text Generation Inference) Endpoint Inference and TGI JSON Grammar Generation (#502)

cpcdoy committed 182 days ago

Verified da8466bd

Parametrizing the sampling evals from the CLI (#926)

clefourrier committed 182 days ago

Verified 52d3d333

Save reasoning_content from litellm as details (#929)

muupan committed 185 days ago

Verified 50756ef2

Add Filipino Benchmark Community Tasks (#852)

ljvmiranda921 committed 185 days ago

Verified d950a1e4

Test for the xet hub team (#928)

clefourrier committed 185 days ago

Verified f586400c

fix tasks list (#906)

alielfilali01 committed 185 days ago

Verified 0da128f5

Fix #878: load tasks before models (#931)

clefourrier committed 185 days ago

Verified 222a31cd

Trackio when available (#930)

NathanHB committed 187 days ago

Verified 0da24778

Upgrade to datasets 4.0 (#924)

NathanHB committed 187 days ago

Verified e53587ef

fix context size check in sglang model (#787)

xgwang committed 187 days ago

Verified 63424f4e

Update translation_literals.py - add romanian (#923)

dianaonutu committed 187 days ago

Verified bab9f6db

Reasoning tags removal is true by default (#925)

clefourrier committed 188 days ago

Verified 4e1bb325

fixes from_model function and adds tests (#921)

NathanHB committed 189 days ago

Verified 4d2bf42c

Restore `avg@4` and `avg@8` for GPQA-D (#922)

lewtun committed 192 days ago

Verified 458a3e96

Added ability to configure concurrent_requests in litellm_model.py (#911)

dameikle committed 193 days ago

Verified b0edbc34

Use lower bound for httpx (#920)

lewtun committed 194 days ago

Verified 76e5affe

making inspect working again (#887)

uralik committed 194 days ago

Verified af52de69

Trying to fix xet bug in test suite (#917)

clefourrier committed 196 days ago

Verified ecef2c66

Older