huggingface/lighteval

Pull Requests Commits
Add subsets for lcb (#587)
plaguss committed 364 days ago
Verified ed084813
adds aime24, 25 and math500 (#586)
NathanHB committed 1 year ago
Verified 4c9af85c
docs: update README to reflect new model evaluation entry points (#581)
czakop committed 1 year ago
Verified 066f84f7
parse seed for vllm (#585)
eldarkurtic committed 1 year ago
Verified 95068aa6
Push details without converting fields to str (#572)
NathanHB committed 1 year ago
Verified 7b421132
Add turkish and word (#583)
bezir committed 1 year ago
Verified bd578a84
Fix vLLM generation with sampling params (#578)
lewtun committed 1 year ago
Verified ebb7377b
Humanity's last exam (#520)
clefourrier committed 1 year ago
Verified 782afe89
Let lighteval support sglang (#552)
Jayon02 committed 1 year ago
Verified 086cf905
raise exception when generation size is more than model length (#571)
NathanHB committed 1 year ago
Verified bee02f7e
Add extended task for LiveCodeBench codegeneration (#548)
plaguss committed 1 year ago
Verified fd479ee6
allows better flexibility for litellm endpoints (#549)
NathanHB committed 1 year ago
Verified d6de1fe2
typo(vllm): `gpu_memory_utilisation` typo (#553)
tpoisonooo committed 1 year ago
Verified fac17bb6
[VLLM] Allows for max tokens to be set in model config file (#547)
NathanHB committed 1 year ago
Verified 78b68abb
fix: broken URLs (#550)
deep-diver committed 1 year ago
Verified da119e81
Fix loading of vllm model from files (#533)
NathanHB committed 1 year ago
Verified d4e6f59b
Fix VLLM data-parallel (#541)
hynky1999 committed 1 year ago
Verified 86f62259
Bug fix extractive match (#540)
hynky1999 committed 1 year ago
Verified 3c9b0c9d
Update README.md (#539)
clefourrier committed 1 year ago
Verified f8405eee
Pass@k (#519)
clefourrier committed 1 year ago
Verified 441d7a4a
Make BLEURT lazy (#536)
hynky1999 committed 1 year ago
Verified 15bdbb81
Add GPQA for instruct models (#534)
lewtun committed 1 year ago
Verified 1ce7331f
Sync Math-verify (#535)
hynky1999 committed 1 year ago
Verified cb35beae
Add custom task (bac-fr) for evaluation of models in french (#518)
mdiazmel committed 1 year ago
Verified d7a1f112
Update french_evals.py
clefourrier committed 1 year ago
Verified be7da176
adds olympiad bench (#521)
NathanHB committed 1 year ago
Verified d332207b
Improve readability of the quick tour. (#501)
vxw3t8fhjsdkghvbdifuk committed 1 year ago
Verified 515bd01d
Implemented the possibility to load predictions from details files and continue evaluating from there (#488)
JoelNiklaus committed 1 year ago
Verified 94fc5a28
add missing inits (#524)
hynky1999 committed 1 year ago
Verified 48d0c284
Math extraction - allow only trying the first match, more customizable latex extraction + bump deps (#522)
hynky1999 committed 1 year ago
Verified 0e462692
Terms Of Use Privacy Policy Refund Policy Impressum