huggingface/lighteval

Pull Requests Commits

clefourrier committed 225 days ago

ee6149e9

one file per eval

clefourrier committed 225 days ago

2114c616

small prompt fix

clefourrier committed 225 days ago

2a108268

added enforce eager

clefourrier committed 225 days ago

6f5d92f1

clefourrier committed 225 days ago

91ad5737

updated prompts/instruction management

clefourrier committed 225 days ago

78a5c238

clefourrier committed 225 days ago

bae95440

Complete TranslationLiterals for Language.ESTONIAN (#779)

spyysalo committed 225 days ago

bc5a5641

Fix GPQA and index extractive metric (#829)

clefourrier committed 225 days ago

031e09ce

focused data on actually existing languages

clefourrier committed 226 days ago

a549107c

clefourrier committed 226 days ago

2852db8d

clefourrier committed 226 days ago

a94973a2

clefourrier committed 226 days ago

ffa43ccb

clefourrier committed 226 days ago

75af52df

added instruct specific evals

clefourrier committed 226 days ago

5b3cd265

Fix system prompt concatenated with instruction in chat template (#830)

clefourrier committed 227 days ago

Verified 0ba8812f

add TUMLU-mini benchmark, solves #577 (#811)

ceferisbarov committed 227 days ago

Verified 936612cb

Add TranslationLiterals for Language.DANISH (#770)

spyysalo committed 227 days ago

Verified b0c092a1

Fix Typos in Documentation and Task Description (#810)

leopardracer committed 227 days ago

Verified a3725503

Catch ROCM/HIP/AMD oom in `should_reduce_batch_size` (#812)

mcleish7 committed 228 days ago

Verified 95ec6691

Flores-200 combinations->permutations (#820)

anton-l committed 228 days ago

Verified dd1af5a9

set default temperature to 0 in generation config (#814)

NathanHB committed 231 days ago

Verified 327071fe

Add org_to_bill parameter to documentation (#781)

tfrere committed 234 days ago

Verified 3d448976

Add Bulgarian and Macedonian literals (#769)

dianaonutu committed 245 days ago

Verified 9771f3a0

Fix: Assign SummaCZS instance to `self.summac` in Faithfulness metric (#795)

sahilds1 committed 245 days ago

Verified 672f6940

[IFEval] Speed up think tag removal (#792)

lewtun committed 247 days ago

Verified 53df8598

add a regex to remove think tags before evaluating ifeval (#791)

NathanHB committed 247 days ago

Verified 8260f590

fix: multiple typos of different value (#782)

DeVikingMark committed 254 days ago

Verified 78871727

Making bootstrap_iters an arg (#697)

pratyushmaini committed 254 days ago

Verified 88050354

Adds GSM-PLUS (#780)

NathanHB committed 254 days ago

Verified 9619194b

Older