huggingface/lighteval

Pull Requests Commits

Merge branch 'main' into clem_fix_templates

clefourrier committed 224 days ago

Verified 65bc25d9

Refacto and remove bloated code (#709)

NathanHB committed 224 days ago

Verified 9288bd84

Fix system prompt concatenated with instruction in chat template (#830)

clefourrier committed 225 days ago

Verified 0ba8812f

fix tests, need to invert condition

clefourrier committed 225 days ago

00988bb8

clefourrier committed 225 days ago

47857e48

better to have a strict extraction for index extraction in general actually

clefourrier committed 225 days ago

9c55b986

Merge branch 'main' into clem_fix_templates

clefourrier committed 225 days ago

Verified b2eb5bc8

fixing whitespace and instruction in prompt

clefourrier committed 225 days ago

5cc26070

too many false positives with the current gpqa metric extraction, making it more string

clefourrier committed 225 days ago

448859c6

add TUMLU-mini benchmark, solves #577 (#811)

ceferisbarov committed 225 days ago

Verified 936612cb

Add TranslationLiterals for Language.DANISH (#770)

spyysalo committed 225 days ago

Verified b0c092a1

Fix Typos in Documentation and Task Description (#810)

leopardracer committed 225 days ago

Verified a3725503

Catch ROCM/HIP/AMD oom in `should_reduce_batch_size` (#812)

mcleish7 committed 226 days ago

Verified 95ec6691

Flores-200 combinations->permutations (#820)

anton-l committed 226 days ago

Verified dd1af5a9

set default temperature to 0 in generation config (#814)

NathanHB committed 229 days ago

Verified 327071fe

Add org_to_bill parameter to documentation (#781)

tfrere committed 232 days ago

Verified 3d448976

Add Bulgarian and Macedonian literals (#769)

dianaonutu committed 243 days ago

Verified 9771f3a0

Fix: Assign SummaCZS instance to `self.summac` in Faithfulness metric (#795)

sahilds1 committed 243 days ago

Verified 672f6940

[IFEval] Speed up think tag removal (#792)

lewtun committed 245 days ago

Verified 53df8598

add a regex to remove think tags before evaluating ifeval (#791)

NathanHB committed 245 days ago

Verified 8260f590

fix: multiple typos of different value (#782)

DeVikingMark committed 252 days ago

Verified 78871727

Making bootstrap_iters an arg (#697)

pratyushmaini committed 252 days ago

Verified 88050354

Adds GSM-PLUS (#780)

NathanHB committed 252 days ago

Verified 9619194b

Bump dev version to 0.10.1.dev0 (#777)

NathanHB committed 257 days ago

Verified 9dc2e538

Async vllm (#693)

clefourrier committed 258 days ago

Verified c4826ea9

Bump ruff version (#774)

NathanHB committed 258 days ago

Verified c9c19e12

Nanotron, Multilingual tasks update + misc (#756)

hynky1999 committed 258 days ago

Verified 034c23b7

add dependencies to run after pip install (#767)

NathanHB committed 259 days ago

Verified 26517504

fix custom model example (#766)

NathanHB committed 259 days ago

Verified cce0bfca

Adds template for custom path saving results (#755)

NathanHB committed 259 days ago

Verified b6816a87

Older