huggingface/lighteval

Pull Requests Commits

Remove dot in test_extractive_match.py

alozowski committed 262 days ago

af31fadb

Fix a typo in test_extractive_match.py

alozowski committed 262 days ago

3e332423

Update custom_yourbench_task_mcq to use multilingual_extractive_match_metric and instruction

alozowski committed 262 days ago

fc82f573

fix extractive match to accept (A) (#746)

hynky1999 committed 262 days ago

Verified 74df0b1c

Merge remote-tracking branch 'origin/main' into mcq-support-yourbench

alozowski committed 263 days ago

c7f4ae90

fix litellm (#736)

NathanHB committed 265 days ago

Verified a5903760

Remove XML prompt format and unused metadata fields

alozowski committed 265 days ago

9c0fff62

Adds More Generative tasks (#694)

hynky1999 committed 265 days ago

Verified c6d1231f

Update custom_yourbench_task_mcq.py to use ExactMatches

alozowski committed 266 days ago

e90c74d3

Apply Ruff to custom_yourbench_task_mcq.py

alozowski committed 266 days ago

df19a810

Add MCQ support to Yourbench evaluation

alozowski committed 266 days ago

9deb4020

Update README.md (#733)

clefourrier committed 266 days ago

Verified f684d359

Fix revision arg for vLLM tokenizer (#721)

lewtun committed 267 days ago

Verified d3da6b9b

Added support for quantization in vLLM backend (#690)

SulRash committed 270 days ago

Verified 04a74a28

Fix tqdm logging (#711)

Vectorrent committed 270 days ago

Verified f7392fa0

add livecodebench v6 (#712)

Cppowboy committed 270 days ago

Verified 7477de09

Patch 0.9.1 (#708)

NathanHB committed 275 days ago

Verified c8714563

Bump dev version0.9.1.dev0 (#705)

NathanHB committed 276 days ago

Verified 20cff959

better release notes (#704)

NathanHB committed 276 days ago

Verified 42c9c61a

fix typos (#702)

omahs committed 276 days ago

Verified 77efde06

Update README.md (#703)

NathanHB committed 276 days ago

Verified 9bf210c0

[FIX] Inference providers (#701)

clefourrier committed 276 days ago

Verified 039debce

docs: improve consistency in punctuation of metric list (#605)

mariagrandury committed 276 days ago

Verified 40626e7a

Update split iteration for DynamicBatchingDataset (#684)

qubvel committed 276 days ago

Verified d146f9b1

Fix TransformersModel.from_model() method (#691)

Vectorrent committed 276 days ago

Verified ea729318

Added custom model inference. (#437)

JoelNiklaus committed 277 days ago

Verified e67ed9c9

reorder authors (#699)

NathanHB committed 277 days ago

Verified dfeb2349

Add pass@1 for GPQA-D and MATH-500 (#698)

lewtun committed 277 days ago

Verified d50bc307

Nathan adds wanddb logging (#685)

NathanHB committed 282 days ago

Verified 96e885d4

fix gsm8k metric (#688)

NathanHB committed 282 days ago

Verified c776694a

Older