huggingface/lighteval

Pull Requests Commits

clefourrier committed 145 days ago

085f59cf

Update src/lighteval/utils/cache_management.py

clefourrier committed 145 days ago

Verified ef5ffe74

this makes little sense

clefourrier committed 145 days ago

668e2f44

clefourrier committed 145 days ago

dbac859f

fixing the metric changed res by 1 point

clefourrier committed 145 days ago

0c4a429c

update vllm test

clefourrier committed 145 days ago

f7c62bbb

adapted to new functions

clefourrier committed 146 days ago

27be046c

simplified system with cleaner task_id

clefourrier committed 146 days ago

b463efff

update caching tests

clefourrier committed 147 days ago

68688b43

clefourrier committed 147 days ago

42ec1ce5

removed token system + added an actual separation between tasks with different hashs + added a separation for different metrics in the same task + has

clefourrier committed 147 days ago

c7d1eb05

last fix is to possibly push to configs

clefourrier committed 148 days ago

a3eeebd5

better str for classes, which allows correct hashing

clefourrier committed 148 days ago

8b28aba1

Merge branch 'main' into test_mmlu_redux_2

clefourrier committed 148 days ago

971e0821

update caching"

clefourrier committed 148 days ago

96df0e7f

Update docs and enforces google's docstring style (#941)

NathanHB committed 148 days ago

Verified 7ed2636e

clefourrier committed 152 days ago

951cbc0c

add default metric for mmlu_redux

clefourrier committed 152 days ago

b0e55846

fix metrics kwargs passing

clefourrier committed 152 days ago

b5975462

Registry refactorisation (#937)

clefourrier committed 155 days ago

Verified 1036929d

reduces cli args redundancy (#932)

NathanHB committed 159 days ago

Verified ee31223d

Fixe Sampling Metrics and Evals (#938)

NathanHB committed 160 days ago

Verified 0b288e3c

Apply suggestion from @NathanHB

NathanHB committed 162 days ago

Verified 1e139ab9

Merge branch 'main' into test_mmlu_redux_2

clefourrier committed 162 days ago

Verified 541c89fd

Fix `TGI` (Text Generation Inference) Endpoint Inference and TGI JSON Grammar Generation (#502)

cpcdoy committed 162 days ago

Verified da8466bd

Parametrizing the sampling evals from the CLI (#926)

clefourrier committed 162 days ago

Verified 52d3d333

Save reasoning_content from litellm as details (#929)

muupan committed 165 days ago

Verified 50756ef2

Add Filipino Benchmark Community Tasks (#852)

ljvmiranda921 committed 165 days ago

Verified d950a1e4

Test for the xet hub team (#928)

clefourrier committed 165 days ago

Verified f586400c

fix tasks list (#906)

alielfilali01 committed 165 days ago

Verified 0da128f5

Older