huggingface/lighteval

Pull Requests Commits

clefourrier committed 1 year ago

085f59cf

Update src/lighteval/utils/cache_management.py

clefourrier committed 1 year ago

Verified ef5ffe74

this makes little sense

clefourrier committed 1 year ago

668e2f44

clefourrier committed 1 year ago

dbac859f

fixing the metric changed res by 1 point

clefourrier committed 1 year ago

0c4a429c

update vllm test

clefourrier committed 1 year ago

f7c62bbb

adapted to new functions

clefourrier committed 1 year ago

27be046c

simplified system with cleaner task_id

clefourrier committed 1 year ago

b463efff

update caching tests

clefourrier committed 1 year ago

68688b43

clefourrier committed 1 year ago

42ec1ce5

removed token system + added an actual separation between tasks with different hashs + added a separation for different metrics in the same task + has

clefourrier committed 1 year ago

c7d1eb05

last fix is to possibly push to configs

clefourrier committed 1 year ago

a3eeebd5

better str for classes, which allows correct hashing

clefourrier committed 1 year ago

8b28aba1

Merge branch 'main' into test_mmlu_redux_2

clefourrier committed 1 year ago

971e0821

update caching"

clefourrier committed 1 year ago

96df0e7f

Update docs and enforces google's docstring style (#941)

NathanHB committed 1 year ago

Verified 7ed2636e

clefourrier committed 1 year ago

951cbc0c

add default metric for mmlu_redux

clefourrier committed 1 year ago

b0e55846

fix metrics kwargs passing

clefourrier committed 1 year ago

b5975462

Registry refactorisation (#937)

clefourrier committed 1 year ago

Verified 1036929d

reduces cli args redundancy (#932)

NathanHB committed 1 year ago

Verified ee31223d

Fixe Sampling Metrics and Evals (#938)

NathanHB committed 1 year ago

Verified 0b288e3c

Apply suggestion from @NathanHB

NathanHB committed 1 year ago

Verified 1e139ab9

Merge branch 'main' into test_mmlu_redux_2

clefourrier committed 1 year ago

Verified 541c89fd

Fix `TGI` (Text Generation Inference) Endpoint Inference and TGI JSON Grammar Generation (#502)

cpcdoy committed 1 year ago

Verified da8466bd

Parametrizing the sampling evals from the CLI (#926)

clefourrier committed 1 year ago

Verified 52d3d333

Save reasoning_content from litellm as details (#929)

muupan committed 1 year ago

Verified 50756ef2

Add Filipino Benchmark Community Tasks (#852)

ljvmiranda921 committed 1 year ago

Verified d950a1e4

Test for the xet hub team (#928)

clefourrier committed 1 year ago

Verified f586400c

fix tasks list (#906)

alielfilali01 committed 1 year ago

Verified 0da128f5

Older