huggingface/lighteval

Pull Requests Commits

clefourrier committed 222 days ago

9789608c

clefourrier committed 222 days ago

0c3039fd

fake model config

clefourrier committed 222 days ago

9cef2759

clefourrier committed 222 days ago

87861887

clefourrier committed 223 days ago

88c5a1bb

tests - will look later at why noqa ignored

clefourrier committed 223 days ago

922fa5aa

only mandatory field is model_name, tests wip

clefourrier committed 223 days ago

831747c2

clefourrier committed 223 days ago

0ad058b0

better management of pydantic to dict for cleaner save

clefourrier committed 223 days ago

4fbdf751

restructuring: removing ModelInfo which is useless, to log instead the full ModelConfig object. Also moved some classes around as they were not in the right place

clefourrier committed 223 days ago

34b1d6ec

Moved some files (#905)

clefourrier committed 223 days ago

Verified d0cd4c91

Debug log likelihood evals which are broken for accelerate (#901)

clefourrier committed 224 days ago

Verified ea1dd184

Number of fixes to run accelerate evaluations (#898)

clefourrier committed 225 days ago

Verified 64f93b0e

Update transformers and vllm dependencies (#899)

clefourrier committed 225 days ago

Verified 865335e4

Added post processing (for reasoning tokens) to pipeline (#882)

clefourrier committed 226 days ago

Verified d7beacbc

fix path concatenation (#891)

clefourrier committed 226 days ago

Verified 994e9e07

Fix DP>1 & TP>1 evals with vllm (#841)

NouamaneTazi committed 229 days ago

Verified 24895519

Adds continuous batching (#850)

NathanHB committed 229 days ago

Verified 99bfd9f2

Fix typo: CONCURENT_CALLS -> CONCURRENT_CALLS (#884)

muupan committed 229 days ago

Verified 1404ba1d

Fixed typo in Python API documentation (#862)

dtung8068 committed 229 days ago

Verified 3b1126ad

Automatically infer whether to use a chat template or not instead of using kwargs (#885)

clefourrier committed 229 days ago

Verified e7d70902

Pin transforers version to fix broken test suite due to mismatch with vllm (#886)

clefourrier committed 230 days ago

Verified bc5e450a

Remove datasets 4 from pyproject (#863)

clefourrier committed 244 days ago

Verified d805f9fa

fix: update python api user docs (#784)

ryantzr1 committed 261 days ago

Verified 88bd36ae

Update extractive_match_utils.py for words where `:` is preceded by a space (#831)

clefourrier committed 265 days ago

Verified 126f908a

Complete TranslationLiterals for Language.ESTONIAN (#779)

spyysalo committed 265 days ago

Verified a455539b

Fix GPQA and index extractive metric (#829)

clefourrier committed 265 days ago

Verified 50c78a2e

Icelandic addition (#775)

joenaess committed 266 days ago

Verified 4dc92474

Refacto and remove bloated code (#709)

NathanHB committed 266 days ago

Verified 9288bd84

Fix system prompt concatenated with instruction in chat template (#830)

clefourrier committed 267 days ago

Verified 0ba8812f

Older