huggingface/lighteval

Pull Requests Commits

Apply suggestions from code review

clefourrier committed 138 days ago

Verified c179876e

Merge branch 'main' into nathan-add-continious-batching

clefourrier committed 138 days ago

Verified 84237ad4

Fix typo: CONCURENT_CALLS -> CONCURRENT_CALLS (#884)

muupan committed 138 days ago

Verified 1404ba1d

Fixed typo in Python API documentation (#862)

dtung8068 committed 138 days ago

Verified 3b1126ad

Automatically infer whether to use a chat template or not instead of using kwargs (#885)

clefourrier committed 138 days ago

Verified e7d70902

Pin transforers version to fix broken test suite due to mismatch with vllm (#886)

clefourrier committed 139 days ago

Verified bc5e450a

Remove datasets 4 from pyproject (#863)

clefourrier committed 153 days ago

Verified d805f9fa

reset vllm model file config

NathanHB committed 163 days ago

7453c6f2

NathanHB committed 163 days ago

8344961f

NathanHB committed 163 days ago

96466e4e

NathanHB committed 163 days ago

fe6f24c4

NathanHB committed 166 days ago

1da56bdd

NathanHB committed 167 days ago

df98d9be

NathanHB committed 170 days ago

0f772b1a

Merge branch 'main' into add-fast-generate

NathanHB committed 170 days ago

Verified 101083e0

fix: update python api user docs (#784)

ryantzr1 committed 170 days ago

Verified 88bd36ae

Update extractive_match_utils.py for words where `:` is preceded by a space (#831)

clefourrier committed 174 days ago

Verified 126f908a

Complete TranslationLiterals for Language.ESTONIAN (#779)

spyysalo committed 174 days ago

Verified a455539b

Fix GPQA and index extractive metric (#829)

clefourrier committed 174 days ago

Verified 50c78a2e

Icelandic addition (#775)

joenaess committed 175 days ago

Verified 4dc92474

Refacto and remove bloated code (#709)

NathanHB committed 175 days ago

Verified 9288bd84

Fix system prompt concatenated with instruction in chat template (#830)

clefourrier committed 176 days ago

Verified 0ba8812f

add TUMLU-mini benchmark, solves #577 (#811)

ceferisbarov committed 176 days ago

Verified 936612cb

Add TranslationLiterals for Language.DANISH (#770)

spyysalo committed 176 days ago

Verified b0c092a1

Fix Typos in Documentation and Task Description (#810)

leopardracer committed 176 days ago

Verified a3725503

Catch ROCM/HIP/AMD oom in `should_reduce_batch_size` (#812)

mcleish7 committed 177 days ago

Verified 95ec6691

Flores-200 combinations->permutations (#820)

anton-l committed 177 days ago

Verified dd1af5a9

set default temperature to 0 in generation config (#814)

NathanHB committed 180 days ago

Verified 327071fe

Add org_to_bill parameter to documentation (#781)

tfrere committed 183 days ago

Verified 3d448976

Merge branch 'main' into add-fast-generate

clefourrier committed 194 days ago

Verified 2b162f7f

Older