huggingface/lighteval

Pull Requests Commits

lewtun committed 249 days ago

7f7134b4

Merge branch 'model_release' into iflores200

lewtun committed 249 days ago

7cea9ffc

Fix mgsm (#849)

lewtun committed 249 days ago

Verified d3116369

Merge branch 'add_mgsm' into model_release

lewtun committed 250 days ago

8cde8d03

Fix input language for Global MMLU (#847)

lewtun committed 251 days ago

Verified 42a61c18

Fix MMLU Pro indexing (#846)

lewtun committed 251 days ago

Verified 4561ee9e

Adding Global MMLU (#837)

clefourrier committed 251 days ago

Verified 486427ed

Revert "Extend judge context (#842)" (#843)

lewtun committed 252 days ago

Verified e8970480

Extend judge context (#842)

lewtun committed 252 days ago

Verified ee03383c

clefourrier committed 252 days ago

Verified 817a1db5

Adding belebele instruct (#836)

clefourrier committed 252 days ago

Verified a394445a

Adding mmlu pro (#835)

clefourrier committed 252 days ago

Verified 87cdf289

changed indices from letters to native letters

clefourrier committed 252 days ago

6d657557

Extend `max_model_len` for local LM judges and range of OpenAI judge models (#840)

lewtun committed 252 days ago

Verified 15b03113

clefourrier committed 252 days ago

5fe6ea0b

added flores200

clefourrier committed 252 days ago

a31750c4

clefourrier committed 252 days ago

657b04a7

clefourrier committed 252 days ago

3a977317

Fix system prompt concatenated with instruction in chat template (#830)

clefourrier committed 255 days ago

Verified 0ba8812f

add TUMLU-mini benchmark, solves #577 (#811)

ceferisbarov committed 255 days ago

Verified 936612cb

Add TranslationLiterals for Language.DANISH (#770)

spyysalo committed 256 days ago

Verified b0c092a1

Fix Typos in Documentation and Task Description (#810)

leopardracer committed 256 days ago

Verified a3725503

Catch ROCM/HIP/AMD oom in `should_reduce_batch_size` (#812)

mcleish7 committed 256 days ago

Verified 95ec6691

Flores-200 combinations->permutations (#820)

anton-l committed 256 days ago

Verified dd1af5a9

set default temperature to 0 in generation config (#814)

NathanHB committed 259 days ago

Verified 327071fe

Add org_to_bill parameter to documentation (#781)

tfrere committed 262 days ago

Verified 3d448976

Add Bulgarian and Macedonian literals (#769)

dianaonutu committed 273 days ago

Verified 9771f3a0

Fix: Assign SummaCZS instance to `self.summac` in Faithfulness metric (#795)

sahilds1 committed 274 days ago

Verified 672f6940

[IFEval] Speed up think tag removal (#792)

lewtun committed 275 days ago

Verified 53df8598

add a regex to remove think tags before evaluating ifeval (#791)

NathanHB committed 275 days ago

Verified 8260f590

Older