huggingface/lighteval

Pull Requests Commits

Merge branch 'main' into think

clefourrier committed 1 year ago

Verified 3f56c4fb

Fix DP>1 & TP>1 evals with vllm (#841)

nmntz committed 1 year ago

Verified 24895519

Adds continuous batching (#850)

NathanHB committed 1 year ago

Verified 99bfd9f2

Fix typo: CONCURENT_CALLS -> CONCURRENT_CALLS (#884)

muupan committed 1 year ago

Verified 1404ba1d

Fixed typo in Python API documentation (#862)

dtung8068 committed 1 year ago

Verified 3b1126ad

Automatically infer whether to use a chat template or not instead of using kwargs (#885)

clefourrier committed 1 year ago

Verified e7d70902

clefourrier committed 1 year ago

94c756df

clefourrier committed 1 year ago

b948696e

Merge branch 'main' into think

clefourrier committed 1 year ago

Verified 8a64c6d9

Pin transforers version to fix broken test suite due to mismatch with vllm (#886)

clefourrier committed 1 year ago

Verified bc5e450a

clefourrier committed 1 year ago

40d04c36

small reorg of lighteval task class to allow mocking

clefourrier committed 1 year ago

17008e8a

more robust reasoning token management

clefourrier committed 1 year ago

6349a791

unrelated, updated doc which was completely outdated

clefourrier committed 1 year ago

18a53968

clefourrier committed 1 year ago

9525a29e

small fix max_len for vllm models

clefourrier committed 1 year ago

4b8c7c5c

add aime avg@64 like in qwen paper

clefourrier committed 1 year ago

4fa53fb1

manage user args

clefourrier committed 1 year ago

7083da44

doc update for reasoning_pair kwarg

clefourrier committed 1 year ago

2ff36268

couple bug fixes with the linter + an actual bug when updating the dataclass values

clefourrier committed 1 year ago

dfa9b01d

fix, tokens should be list of tuples

clefourrier committed 1 year ago

772e9f71

clefourrier committed 1 year ago

33ae811b

clefourrier committed 1 year ago

90f69692

default factory for dataclass

clefourrier committed 1 year ago

d32d6836

added post processing

clefourrier committed 1 year ago

44bd18ff

Remove datasets 4 from pyproject (#863)

clefourrier committed 1 year ago

Verified d805f9fa

fix: update python api user docs (#784)

ryantzr1 committed 1 year ago

Verified 88bd36ae

Update extractive_match_utils.py for words where `:` is preceded by a space (#831)

clefourrier committed 1 year ago

Verified 126f908a

Complete TranslationLiterals for Language.ESTONIAN (#779)

spyysalo committed 1 year ago

Verified a455539b

Fix GPQA and index extractive metric (#829)

clefourrier committed 1 year ago

Verified 50c78a2e

Older