huggingface/lighteval

Pull Requests Commits

Merge branch 'main' into think

clefourrier committed 230 days ago

Verified 3f56c4fb

Fix DP>1 & TP>1 evals with vllm (#841)

NouamaneTazi committed 233 days ago

Verified 24895519

Adds continuous batching (#850)

NathanHB committed 233 days ago

Verified 99bfd9f2

Fix typo: CONCURENT_CALLS -> CONCURRENT_CALLS (#884)

muupan committed 233 days ago

Verified 1404ba1d

Fixed typo in Python API documentation (#862)

dtung8068 committed 233 days ago

Verified 3b1126ad

Automatically infer whether to use a chat template or not instead of using kwargs (#885)

clefourrier committed 233 days ago

Verified e7d70902

clefourrier committed 233 days ago

94c756df

clefourrier committed 233 days ago

b948696e

Merge branch 'main' into think

clefourrier committed 233 days ago

Verified 8a64c6d9

Pin transforers version to fix broken test suite due to mismatch with vllm (#886)

clefourrier committed 233 days ago

Verified bc5e450a

clefourrier committed 233 days ago

40d04c36

small reorg of lighteval task class to allow mocking

clefourrier committed 233 days ago

17008e8a

more robust reasoning token management

clefourrier committed 233 days ago

6349a791

unrelated, updated doc which was completely outdated

clefourrier committed 235 days ago

18a53968

clefourrier committed 235 days ago

9525a29e

small fix max_len for vllm models

clefourrier committed 235 days ago

4b8c7c5c

add aime avg@64 like in qwen paper

clefourrier committed 236 days ago

4fa53fb1

manage user args

clefourrier committed 237 days ago

7083da44

doc update for reasoning_pair kwarg

clefourrier committed 237 days ago

2ff36268

couple bug fixes with the linter + an actual bug when updating the dataclass values

clefourrier committed 240 days ago

dfa9b01d

fix, tokens should be list of tuples

clefourrier committed 240 days ago

772e9f71

clefourrier committed 240 days ago

33ae811b

clefourrier committed 240 days ago

90f69692

default factory for dataclass

clefourrier committed 240 days ago

d32d6836

added post processing

clefourrier committed 240 days ago

44bd18ff

Remove datasets 4 from pyproject (#863)

clefourrier committed 248 days ago

Verified d805f9fa

fix: update python api user docs (#784)

ryantzr1 committed 265 days ago

Verified 88bd36ae

Update extractive_match_utils.py for words where `:` is preceded by a space (#831)

clefourrier committed 269 days ago

Verified 126f908a

Complete TranslationLiterals for Language.ESTONIAN (#779)

spyysalo committed 269 days ago

Verified a455539b

Fix GPQA and index extractive metric (#829)

clefourrier committed 269 days ago

Verified 50c78a2e

Older