huggingface/lighteval

Pull Requests Commits

remove logging in slow tests and log nvidia smi

NathanHB committed 182 days ago

c9f1c246

NathanHB committed 182 days ago

a427e8da

Track parquet files with Git LFS

NathanHB committed 182 days ago

45eeaf59

Refine acknowledgements in README.md (#976)

clefourrier committed 183 days ago

Verified e61b868a

Support local GGUF in VLLM and use HF tokenizer #943 (#972)

JIElite committed 183 days ago

Verified d6a65ca5

Add auto tests for metrics (#939)

NathanHB committed 184 days ago

Verified 16318bb4

Add IFBench (#944)

clefourrier committed 185 days ago

Verified 3a71f688

Added `backend_options` parameter to llm judges. (#963)

rolshoven committed 185 days ago

Verified 9ba430f1

Multilingual extractiveness (#956)

rolshoven committed 185 days ago

Verified 2dc17885

Fixed bug that prevented the metrics from being mixed (some batched and others not batched) (#958)

rolshoven committed 186 days ago

Verified 96a6882e

Update readme (#961)

NathanHB committed 186 days ago

Verified c409f3f9

set upper bound on vllm version (#964)

NathanHB committed 186 days ago

Verified fc50ca09

MMLU Redux and Fixing the caching (#883)

clefourrier committed 190 days ago

Verified eda7eed3

Update docs and enforces google's docstring style (#941)

NathanHB committed 193 days ago

Verified 7ed2636e

Registry refactorisation (#937)

clefourrier committed 200 days ago

Verified 1036929d

reduces cli args redundancy (#932)

NathanHB committed 204 days ago

Verified ee31223d

Fixe Sampling Metrics and Evals (#938)

NathanHB committed 205 days ago

Verified 0b288e3c

Fix `TGI` (Text Generation Inference) Endpoint Inference and TGI JSON Grammar Generation (#502)

cpcdoy committed 207 days ago

Verified da8466bd

Parametrizing the sampling evals from the CLI (#926)

clefourrier committed 207 days ago

Verified 52d3d333

Save reasoning_content from litellm as details (#929)

muupan committed 210 days ago

Verified 50756ef2

Add Filipino Benchmark Community Tasks (#852)

ljvmiranda921 committed 210 days ago

Verified d950a1e4

Test for the xet hub team (#928)

clefourrier committed 210 days ago

Verified f586400c

fix tasks list (#906)

alielfilali01 committed 210 days ago

Verified 0da128f5

Fix #878: load tasks before models (#931)

clefourrier committed 210 days ago

Verified 222a31cd

Trackio when available (#930)

NathanHB committed 212 days ago

Verified 0da24778

Upgrade to datasets 4.0 (#924)

NathanHB committed 212 days ago

Verified e53587ef

fix context size check in sglang model (#787)

xgwang committed 212 days ago

Verified 63424f4e

Update translation_literals.py - add romanian (#923)

dianaonutu committed 212 days ago

Verified bab9f6db

Reasoning tags removal is true by default (#925)

clefourrier committed 213 days ago

Verified 4e1bb325

fixes from_model function and adds tests (#921)

NathanHB committed 214 days ago

Verified 4d2bf42c

Older