huggingface/lighteval

Pull Requests Commits

Apply suggestions from code review

clefourrier committed 230 days ago

Verified c179876e

Merge branch 'main' into nathan-add-continious-batching

clefourrier committed 230 days ago

Verified 84237ad4

Fix typo: CONCURENT_CALLS -> CONCURRENT_CALLS (#884)

muupan committed 230 days ago

Verified 1404ba1d

Fixed typo in Python API documentation (#862)

dtung8068 committed 230 days ago

Verified 3b1126ad

Automatically infer whether to use a chat template or not instead of using kwargs (#885)

clefourrier committed 230 days ago

Verified e7d70902

Pin transforers version to fix broken test suite due to mismatch with vllm (#886)

clefourrier committed 231 days ago

Verified bc5e450a

Remove datasets 4 from pyproject (#863)

clefourrier committed 245 days ago

Verified d805f9fa

reset vllm model file config

NathanHB committed 255 days ago

7453c6f2

NathanHB committed 255 days ago

8344961f

NathanHB committed 255 days ago

96466e4e

NathanHB committed 255 days ago

fe6f24c4

NathanHB committed 258 days ago

1da56bdd

NathanHB committed 259 days ago

df98d9be

NathanHB committed 262 days ago

0f772b1a

Merge branch 'main' into add-fast-generate

NathanHB committed 262 days ago

Verified 101083e0

fix: update python api user docs (#784)

ryantzr1 committed 262 days ago

Verified 88bd36ae

Update extractive_match_utils.py for words where `:` is preceded by a space (#831)

clefourrier committed 266 days ago

Verified 126f908a

Complete TranslationLiterals for Language.ESTONIAN (#779)

spyysalo committed 266 days ago

Verified a455539b

Fix GPQA and index extractive metric (#829)

clefourrier committed 266 days ago

Verified 50c78a2e

Icelandic addition (#775)

joenaess committed 267 days ago

Verified 4dc92474

Refacto and remove bloated code (#709)

NathanHB committed 267 days ago

Verified 9288bd84

Fix system prompt concatenated with instruction in chat template (#830)

clefourrier committed 268 days ago

Verified 0ba8812f

add TUMLU-mini benchmark, solves #577 (#811)

ceferisbarov committed 268 days ago

Verified 936612cb

Add TranslationLiterals for Language.DANISH (#770)

spyysalo committed 268 days ago

Verified b0c092a1

Fix Typos in Documentation and Task Description (#810)

leopardracer committed 268 days ago

Verified a3725503

Catch ROCM/HIP/AMD oom in `should_reduce_batch_size` (#812)

mcleish7 committed 269 days ago

Verified 95ec6691

Flores-200 combinations->permutations (#820)

anton-l committed 269 days ago

Verified dd1af5a9

set default temperature to 0 in generation config (#814)

NathanHB committed 272 days ago

Verified 327071fe

Add org_to_bill parameter to documentation (#781)

tfrere committed 275 days ago

Verified 3d448976

Merge branch 'main' into add-fast-generate

clefourrier committed 286 days ago

Verified 2b162f7f

Older