huggingface/lighteval

Pull Requests Commits

clefourrier committed 165 days ago

085f59cf

Update src/lighteval/utils/cache_management.py

clefourrier committed 165 days ago

Verified ef5ffe74

this makes little sense

clefourrier committed 166 days ago

668e2f44

clefourrier committed 166 days ago

dbac859f

fixing the metric changed res by 1 point

clefourrier committed 166 days ago

0c4a429c

update vllm test

clefourrier committed 166 days ago

f7c62bbb

adapted to new functions

clefourrier committed 166 days ago

27be046c

simplified system with cleaner task_id

clefourrier committed 166 days ago

b463efff

update caching tests

clefourrier committed 167 days ago

68688b43

clefourrier committed 167 days ago

42ec1ce5

removed token system + added an actual separation between tasks with different hashs + added a separation for different metrics in the same task + has

clefourrier committed 167 days ago

c7d1eb05

last fix is to possibly push to configs

clefourrier committed 168 days ago

a3eeebd5

better str for classes, which allows correct hashing

clefourrier committed 168 days ago

8b28aba1

Merge branch 'main' into test_mmlu_redux_2

clefourrier committed 168 days ago

971e0821

update caching"

clefourrier committed 168 days ago

96df0e7f

Update docs and enforces google's docstring style (#941)

NathanHB committed 169 days ago

Verified 7ed2636e

clefourrier committed 172 days ago

951cbc0c

add default metric for mmlu_redux

clefourrier committed 172 days ago

b0e55846

fix metrics kwargs passing

clefourrier committed 172 days ago

b5975462

Registry refactorisation (#937)

clefourrier committed 175 days ago

Verified 1036929d

reduces cli args redundancy (#932)

NathanHB committed 180 days ago

Verified ee31223d

Fixe Sampling Metrics and Evals (#938)

NathanHB committed 181 days ago

Verified 0b288e3c

Apply suggestion from @NathanHB

NathanHB committed 183 days ago

Verified 1e139ab9

Merge branch 'main' into test_mmlu_redux_2

clefourrier committed 183 days ago

Verified 541c89fd

Fix `TGI` (Text Generation Inference) Endpoint Inference and TGI JSON Grammar Generation (#502)

cpcdoy committed 183 days ago

Verified da8466bd

Parametrizing the sampling evals from the CLI (#926)

clefourrier committed 183 days ago

Verified 52d3d333

Save reasoning_content from litellm as details (#929)

muupan committed 186 days ago

Verified 50756ef2

Add Filipino Benchmark Community Tasks (#852)

ljvmiranda921 committed 186 days ago

Verified d950a1e4

Test for the xet hub team (#928)

clefourrier committed 186 days ago

Verified f586400c

fix tasks list (#906)

alielfilali01 committed 186 days ago

Verified 0da128f5

Older