huggingface/lighteval

Pull Requests Commits

Merge branch 'main' into think

clefourrier committed 206 days ago

Verified 3f56c4fb

Fix DP>1 & TP>1 evals with vllm (#841)

NouamaneTazi committed 209 days ago

Verified 24895519

Adds continuous batching (#850)

NathanHB committed 209 days ago

Verified 99bfd9f2

Fix typo: CONCURENT_CALLS -> CONCURRENT_CALLS (#884)

muupan committed 209 days ago

Verified 1404ba1d

Fixed typo in Python API documentation (#862)

dtung8068 committed 209 days ago

Verified 3b1126ad

Automatically infer whether to use a chat template or not instead of using kwargs (#885)

clefourrier committed 209 days ago

Verified e7d70902

clefourrier committed 210 days ago

94c756df

clefourrier committed 210 days ago

b948696e

Merge branch 'main' into think

clefourrier committed 210 days ago

Verified 8a64c6d9

Pin transforers version to fix broken test suite due to mismatch with vllm (#886)

clefourrier committed 210 days ago

Verified bc5e450a

clefourrier committed 210 days ago

40d04c36

small reorg of lighteval task class to allow mocking

clefourrier committed 210 days ago

17008e8a

more robust reasoning token management

clefourrier committed 210 days ago

6349a791

unrelated, updated doc which was completely outdated

clefourrier committed 212 days ago

18a53968

clefourrier committed 212 days ago

9525a29e

small fix max_len for vllm models

clefourrier committed 212 days ago

4b8c7c5c

add aime avg@64 like in qwen paper

clefourrier committed 213 days ago

4fa53fb1

manage user args

clefourrier committed 213 days ago

7083da44

doc update for reasoning_pair kwarg

clefourrier committed 213 days ago

2ff36268

couple bug fixes with the linter + an actual bug when updating the dataclass values

clefourrier committed 216 days ago

dfa9b01d

fix, tokens should be list of tuples

clefourrier committed 216 days ago

772e9f71

clefourrier committed 216 days ago

33ae811b

clefourrier committed 216 days ago

90f69692

default factory for dataclass

clefourrier committed 216 days ago

d32d6836

added post processing

clefourrier committed 216 days ago

44bd18ff

Remove datasets 4 from pyproject (#863)

clefourrier committed 224 days ago

Verified d805f9fa

fix: update python api user docs (#784)

ryantzr1 committed 241 days ago

Verified 88bd36ae

Update extractive_match_utils.py for words where `:` is preceded by a space (#831)

clefourrier committed 245 days ago

Verified 126f908a

Complete TranslationLiterals for Language.ESTONIAN (#779)

spyysalo committed 245 days ago

Verified a455539b

Fix GPQA and index extractive metric (#829)

clefourrier committed 245 days ago

Verified 50c78a2e

Older