huggingface/lighteval

Pull Requests Commits

changed indices from letters to native letters

clefourrier committed 171 days ago

6d657557

clefourrier committed 171 days ago

657b04a7

clefourrier committed 171 days ago

3a977317

Fix system prompt concatenated with instruction in chat template (#830)

clefourrier committed 174 days ago

Verified 0ba8812f

add TUMLU-mini benchmark, solves #577 (#811)

ceferisbarov committed 174 days ago

Verified 936612cb

Add TranslationLiterals for Language.DANISH (#770)

spyysalo committed 174 days ago

Verified b0c092a1

Fix Typos in Documentation and Task Description (#810)

leopardracer committed 174 days ago

Verified a3725503

Catch ROCM/HIP/AMD oom in `should_reduce_batch_size` (#812)

mcleish7 committed 174 days ago

Verified 95ec6691

Flores-200 combinations->permutations (#820)

anton-l committed 175 days ago

Verified dd1af5a9

set default temperature to 0 in generation config (#814)

NathanHB committed 178 days ago

Verified 327071fe

Add org_to_bill parameter to documentation (#781)

tfrere committed 181 days ago

Verified 3d448976

Add Bulgarian and Macedonian literals (#769)

dianaonutu committed 192 days ago

Verified 9771f3a0

Fix: Assign SummaCZS instance to `self.summac` in Faithfulness metric (#795)

sahilds1 committed 192 days ago

Verified 672f6940

[IFEval] Speed up think tag removal (#792)

lewtun committed 194 days ago

Verified 53df8598

add a regex to remove think tags before evaluating ifeval (#791)

NathanHB committed 194 days ago

Verified 8260f590

fix: multiple typos of different value (#782)

DeVikingMark committed 200 days ago

Verified 78871727

Making bootstrap_iters an arg (#697)

pratyushmaini committed 200 days ago

Verified 88050354

Adds GSM-PLUS (#780)

NathanHB committed 201 days ago

Verified 9619194b

Bump dev version to 0.10.1.dev0 (#777)

NathanHB committed 206 days ago

Verified 9dc2e538

Async vllm (#693)

clefourrier committed 207 days ago

Verified c4826ea9

Bump ruff version (#774)

NathanHB committed 207 days ago

Verified c9c19e12

Nanotron, Multilingual tasks update + misc (#756)

hynky1999 committed 207 days ago

Verified 034c23b7

add dependencies to run after pip install (#767)

NathanHB committed 208 days ago

Verified 26517504

fix custom model example (#766)

NathanHB committed 208 days ago

Verified cce0bfca

Adds template for custom path saving results (#755)

NathanHB committed 208 days ago

Verified b6816a87

Allow for model kwargs when loading transformers from pretrained (#754)

NathanHB committed 208 days ago

Verified ce1dbb5e

Add MCQ support to Yourbench evaluation (#734)

alozowski committed 209 days ago

Verified 317cb504

Fix task metric type mismatch (#743)

Abelgurung committed 209 days ago

Verified 3bb8a50f

Adds multimodal support and MMMU pro (#675)

NathanHB committed 209 days ago

Verified 1607dc10

Added Flores 200 (#717)

clefourrier committed 210 days ago

Verified 63be4b0f

Older