huggingface/lighteval

Pull Requests Commits

fixes for ruler

NathanHB committed 197 days ago

df77d217

fix tokenizer vllm

eliebak committed 201 days ago

2d5b7fe1

Merge commit '0ba8812' into ruler-env-correct-sys

eliebak committed 201 days ago

300be672

eliebak committed 201 days ago

c32e40d8

Fix system prompt concatenated with instruction in chat template (#830)

clefourrier committed 203 days ago

Verified 0ba8812f

add TUMLU-mini benchmark, solves #577 (#811)

ceferisbarov committed 203 days ago

Verified 936612cb

Add TranslationLiterals for Language.DANISH (#770)

spyysalo committed 203 days ago

Verified b0c092a1

Fix Typos in Documentation and Task Description (#810)

leopardracer committed 203 days ago

Verified a3725503

Catch ROCM/HIP/AMD oom in `should_reduce_batch_size` (#812)

mcleish7 committed 204 days ago

Verified 95ec6691

Flores-200 combinations->permutations (#820)

anton-l committed 204 days ago

Verified dd1af5a9

set default temperature to 0 in generation config (#814)

NathanHB committed 207 days ago

Verified 327071fe

making it flexible with env vriable

eliebak committed 209 days ago

8a97a348

put tuler in extedned tasks

NathanHB committed 209 days ago

57f29215

Add org_to_bill parameter to documentation (#781)

tfrere committed 210 days ago

Verified 3d448976

Add Bulgarian and Macedonian literals (#769)

dianaonutu committed 221 days ago

Verified 9771f3a0

Fix: Assign SummaCZS instance to `self.summac` in Faithfulness metric (#795)

sahilds1 committed 221 days ago

Verified 672f6940

[IFEval] Speed up think tag removal (#792)

lewtun committed 223 days ago

Verified 53df8598

add a regex to remove think tags before evaluating ifeval (#791)

NathanHB committed 223 days ago

Verified 8260f590

eliebak committed 225 days ago

461b8cbf

fix: multiple typos of different value (#782)

DeVikingMark committed 229 days ago

Verified 78871727

Making bootstrap_iters an arg (#697)

pratyushmaini committed 229 days ago

Verified 88050354

Merge branch 'main' into nathan-adds-helet

NathanHB committed 230 days ago

Verified 775705c5

Adds GSM-PLUS (#780)

NathanHB committed 230 days ago

Verified 9619194b

Bump dev version to 0.10.1.dev0 (#777)

NathanHB committed 235 days ago

Verified 9dc2e538

Async vllm (#693)

clefourrier committed 236 days ago

Verified c4826ea9

Bump ruff version (#774)

NathanHB committed 236 days ago

Verified c9c19e12

Nanotron, Multilingual tasks update + misc (#756)

hynky1999 committed 236 days ago

Verified 034c23b7

Update src/lighteval/models/vllm/vllm_model.py

NathanHB committed 237 days ago

Verified a1aee681

Update src/lighteval/tasks/default_prompts.py

NathanHB committed 237 days ago

Verified 248bb678

Merge branch 'main' into nathan-adds-helet

NathanHB committed 237 days ago

Verified a4394ad7

Older