huggingface/lighteval

Pull Requests Commits

update tasks, fix tests for templates, add math_qa template, fix tranlsation metrics

hynky1999 committed 214 days ago

f17858da

updated templates

hynky1999 committed 215 days ago

cb6eb13e

add new translation literals

hynky1999 committed 215 days ago

25449ac8

Async vllm (#693)

clefourrier committed 216 days ago

Verified c4826ea9

Bump ruff version (#774)

NathanHB committed 216 days ago

Verified c9c19e12

Nanotron, Multilingual tasks update + misc (#756)

hynky1999 committed 216 days ago

Verified 034c23b7

add dependencies to run after pip install (#767)

NathanHB committed 217 days ago

Verified 26517504

fix custom model example (#766)

NathanHB committed 217 days ago

Verified cce0bfca

Adds template for custom path saving results (#755)

NathanHB committed 217 days ago

Verified b6816a87

Allow for model kwargs when loading transformers from pretrained (#754)

NathanHB committed 217 days ago

Verified ce1dbb5e

Add MCQ support to Yourbench evaluation (#734)

alozowski committed 218 days ago

Verified 317cb504

Fix task metric type mismatch (#743)

Abelgurung committed 218 days ago

Verified 3bb8a50f

Adds multimodal support and MMMU pro (#675)

NathanHB committed 219 days ago

Verified 1607dc10

Added Flores 200 (#717)

clefourrier committed 219 days ago

Verified 63be4b0f

Update main_endpoint.py (#739)

clefourrier committed 219 days ago

Verified d18f11ab

fix litellm (#736)

NathanHB committed 222 days ago

Verified a5903760

Adds More Generative tasks (#694)

hynky1999 committed 222 days ago

Verified c6d1231f

Update README.md (#733)

clefourrier committed 223 days ago

Verified f684d359

Fix revision arg for vLLM tokenizer (#721)

lewtun committed 223 days ago

Verified d3da6b9b

Added support for quantization in vLLM backend (#690)

SulRash committed 226 days ago

Verified 04a74a28

Fix tqdm logging (#711)

Vectorrent committed 226 days ago

Verified f7392fa0

add livecodebench v6 (#712)

Cppowboy committed 226 days ago

Verified 7477de09

Patch 0.9.1 (#708)

NathanHB committed 231 days ago

Verified c8714563

Bump dev version0.9.1.dev0 (#705)

NathanHB committed 232 days ago

Verified 20cff959

better release notes (#704)

NathanHB committed 232 days ago

Verified 42c9c61a

fix typos (#702)

omahs committed 232 days ago

Verified 77efde06

Update README.md (#703)

NathanHB committed 232 days ago

Verified 9bf210c0

[FIX] Inference providers (#701)

clefourrier committed 232 days ago

Verified 039debce

docs: improve consistency in punctuation of metric list (#605)

mariagrandury committed 232 days ago

Verified 40626e7a

Update split iteration for DynamicBatchingDataset (#684)

qubvel committed 233 days ago

Verified d146f9b1

Older