huggingface/lighteval

Pull Requests Commits

Apply suggestions from code review

clefourrier committed 236 days ago

Verified c179876e

Merge branch 'main' into nathan-add-continious-batching

clefourrier committed 236 days ago

Verified 84237ad4

Fix typo: CONCURENT_CALLS -> CONCURRENT_CALLS (#884)

muupan committed 236 days ago

Verified 1404ba1d

Fixed typo in Python API documentation (#862)

dtung8068 committed 236 days ago

Verified 3b1126ad

Automatically infer whether to use a chat template or not instead of using kwargs (#885)

clefourrier committed 236 days ago

Verified e7d70902

Pin transforers version to fix broken test suite due to mismatch with vllm (#886)

clefourrier committed 237 days ago

Verified bc5e450a

Remove datasets 4 from pyproject (#863)

clefourrier committed 251 days ago

Verified d805f9fa

reset vllm model file config

NathanHB committed 261 days ago

7453c6f2

NathanHB committed 261 days ago

8344961f

NathanHB committed 261 days ago

96466e4e

NathanHB committed 261 days ago

fe6f24c4

NathanHB committed 264 days ago

1da56bdd

NathanHB committed 265 days ago

df98d9be

NathanHB committed 268 days ago

0f772b1a

Merge branch 'main' into add-fast-generate

NathanHB committed 268 days ago

Verified 101083e0

fix: update python api user docs (#784)

ryantzr1 committed 268 days ago

Verified 88bd36ae

Update extractive_match_utils.py for words where `:` is preceded by a space (#831)

clefourrier committed 272 days ago

Verified 126f908a

Complete TranslationLiterals for Language.ESTONIAN (#779)

spyysalo committed 272 days ago

Verified a455539b

Fix GPQA and index extractive metric (#829)

clefourrier committed 272 days ago

Verified 50c78a2e

Icelandic addition (#775)

joenaess committed 273 days ago

Verified 4dc92474

Refacto and remove bloated code (#709)

NathanHB committed 273 days ago

Verified 9288bd84

Fix system prompt concatenated with instruction in chat template (#830)

clefourrier committed 274 days ago

Verified 0ba8812f

add TUMLU-mini benchmark, solves #577 (#811)

ceferisbarov committed 274 days ago

Verified 936612cb

Add TranslationLiterals for Language.DANISH (#770)

spyysalo committed 274 days ago

Verified b0c092a1

Fix Typos in Documentation and Task Description (#810)

leopardracer committed 274 days ago

Verified a3725503

Catch ROCM/HIP/AMD oom in `should_reduce_batch_size` (#812)

mcleish7 committed 275 days ago

Verified 95ec6691

Flores-200 combinations->permutations (#820)

anton-l committed 275 days ago

Verified dd1af5a9

set default temperature to 0 in generation config (#814)

NathanHB committed 278 days ago

Verified 327071fe

Add org_to_bill parameter to documentation (#781)

tfrere committed 281 days ago

Verified 3d448976

Merge branch 'main' into add-fast-generate

clefourrier committed 292 days ago

Verified 2b162f7f

Older