MMLU Redux and Fixing the caching #883
clefourrier
merged
26 commits into
main
from
test_mmlu_redux_2
init
d9de61d9
clefourrier
commented
on 2025-07-25
Update src/lighteval/tasks/default_prompts.py
a2b1bad2
Merge branch 'main' into test_mmlu_redux_2
0e009378
small fixes
8429f5fb
Merge branch 'main' into test_mmlu_redux_2
1cfa2f3f
Merge branch 'main' into test_mmlu_redux_2
f600c40a
Merge branch 'main' into test_mmlu_redux_2
541c89fd
NathanHB
commented
on 2025-08-25
Apply suggestion from @NathanHB
1e139ab9
lewtun
approved these changes
on 2025-08-28
fix metrics kwargs passing
b5975462
add default metric for mmlu_redux
b0e55846
fix
951cbc0c
update caching"
96df0e7f
Merge branch 'main' into test_mmlu_redux_2
971e0821
better str for classes, which allows correct hashing
8b28aba1
last fix is to possibly push to configs
a3eeebd5
removed token system + added an actual separation between tasks with …
c7d1eb05
fix
42ec1ce5
update caching tests
68688b43
simplified system with cleaner task_id
b463efff
adapted to new functions
27be046c
update vllm test
f7c62bbb
fixing the metric changed res by 1 point
0c4a429c
byteorder arg
dbac859f
this makes little sense
668e2f44
NathanHB
commented
on 2025-09-11
NathanHB
commented
on 2025-09-11
NathanHB
commented
on 2025-09-11
NathanHB
approved these changes
on 2025-09-11
Update src/lighteval/utils/cache_management.py
ef5ffe74
comments
085f59cf
Login to write a write a comment.
Login via GitHub