huggingface/lighteval

Pull Requests Commits

clefourrier committed 359 days ago

2ddae377

clefourrier committed 359 days ago

3c68f4d9

clefourrier committed 363 days ago

c0fb8a8b

clefourrier committed 363 days ago

e6a073f1

Fix steps numbers in custom_yourbench_task.py (#643)

alozowski committed 364 days ago

Verified 2583a0a1

Propagate vLLM batch size controls (#588)

alvin319 committed 1 year ago

Verified bbbdd225

Update README.md (#639)

NathanHB committed 1 year ago

Verified bd9019d6

fix:change multichoice_continuations_start_space category (#610)

AnikiFan committed 1 year ago

Verified d9d90862

enable together models and reasoning models as judges. (#537)

JoelNiklaus committed 1 year ago

Verified d1fcd224

Fixing some silent bugs in Arabic Custom Tasks (#556)

alielfilali01 committed 1 year ago

Verified 706f97eb

support more prequantized models (#620)

DerekLiu35 committed 1 year ago

Verified 35d73d5b

v0.8.1.dev (#637)

NathanHB committed 1 year ago

Verified ecb72067

Create custom task (#621)

NathanHB committed 1 year ago

Verified dcb0ab54

log model config (#627)

NathanHB committed 1 year ago

Verified 9c05a83a

relax deps for tgi (#626)

NathanHB committed 1 year ago

Verified 14d289e3

vllm fix sampling params (#625)

NathanHB committed 1 year ago

Verified d46f2f78

relax spacy import to relax dep (#622)

clefourrier committed 1 year ago

Verified 0ae8136d

adds inference providers support (#616)

NathanHB committed 1 year ago

Verified b9a2e567

VLLM + Math-Verify fixes (#603)

hynky1999 committed 1 year ago

Verified d81bafd6

Fixing backend error in main_sglang. (#597)

TankNee committed 1 year ago

Verified f2ddc520

Add subsets for lcb (#587)

plaguss committed 1 year ago

Verified ed084813

adds aime24, 25 and math500 (#586)

NathanHB committed 1 year ago

Verified 4c9af85c

docs: update README to reflect new model evaluation entry points (#581)

czakop committed 1 year ago

Verified 066f84f7

parse seed for vllm (#585)

eldarkurtic committed 1 year ago

Verified 95068aa6

Push details without converting fields to str (#572)

NathanHB committed 1 year ago

Verified 7b421132

Add turkish and word (#583)

bezir committed 1 year ago

Verified bd578a84

Fix vLLM generation with sampling params (#578)

lewtun committed 1 year ago

Verified ebb7377b

Humanity's last exam (#520)

clefourrier committed 1 year ago

Verified 782afe89

Let lighteval support sglang (#552)

Jayon02 committed 1 year ago

Verified 086cf905

raise exception when generation size is more than model length (#571)

NathanHB committed 1 year ago

Verified bee02f7e

Older