huggingface/lighteval

Pull Requests Commits

Merge branch 'model_release' into gmmlu

lewtun committed 169 days ago

Verified c9eda9bc

Revert "Extend judge context (#842)" (#843)

lewtun committed 170 days ago

Verified e8970480

Extend judge context (#842)

lewtun committed 170 days ago

Verified ee03383c

fix letters indices extraction

clefourrier committed 170 days ago

09ae0fbb

Merge branch 'model_release' into gmmlu

clefourrier committed 170 days ago

1152421d

Adding belebele instruct (#836)

clefourrier committed 170 days ago

Verified a394445a

managed closure

clefourrier committed 170 days ago

8b46e91a

Adding mmlu pro (#835)

clefourrier committed 170 days ago

Verified 87cdf289

changed indices to letters, not native letters

clefourrier committed 170 days ago

4f25dda2

Extend `max_model_len` for local LM judges and range of OpenAI judge models (#840)

lewtun committed 170 days ago

Verified 15b03113

typo in prompt, new fix in extractive match

clefourrier committed 170 days ago

ab598d16

clefourrier committed 170 days ago

853db693

clefourrier committed 170 days ago

95cd5e1a

Fix system prompt concatenated with instruction in chat template (#830)

clefourrier committed 173 days ago

Verified 0ba8812f

add TUMLU-mini benchmark, solves #577 (#811)

ceferisbarov committed 173 days ago

Verified 936612cb

Add TranslationLiterals for Language.DANISH (#770)

spyysalo committed 173 days ago

Verified b0c092a1

Fix Typos in Documentation and Task Description (#810)

leopardracer committed 173 days ago

Verified a3725503

Catch ROCM/HIP/AMD oom in `should_reduce_batch_size` (#812)

mcleish7 committed 174 days ago

Verified 95ec6691

Flores-200 combinations->permutations (#820)

anton-l committed 174 days ago

Verified dd1af5a9

set default temperature to 0 in generation config (#814)

NathanHB committed 177 days ago

Verified 327071fe

Add org_to_bill parameter to documentation (#781)

tfrere committed 180 days ago

Verified 3d448976

Add Bulgarian and Macedonian literals (#769)

dianaonutu committed 191 days ago

Verified 9771f3a0

Fix: Assign SummaCZS instance to `self.summac` in Faithfulness metric (#795)

sahilds1 committed 191 days ago

Verified 672f6940

[IFEval] Speed up think tag removal (#792)

lewtun committed 193 days ago

Verified 53df8598

add a regex to remove think tags before evaluating ifeval (#791)

NathanHB committed 193 days ago

Verified 8260f590

fix: multiple typos of different value (#782)

DeVikingMark committed 200 days ago

Verified 78871727

Making bootstrap_iters an arg (#697)

pratyushmaini committed 200 days ago

Verified 88050354

Adds GSM-PLUS (#780)

NathanHB committed 200 days ago

Verified 9619194b

Bump dev version to 0.10.1.dev0 (#777)

NathanHB committed 205 days ago

Verified 9dc2e538

Async vllm (#693)

clefourrier committed 206 days ago

Verified c4826ea9

Older