huggingface/lighteval

Pull Requests Commits

clefourrier committed 300 days ago

64b23f4c

Update src/lighteval/tasks/registry.py

clefourrier committed 300 days ago

Verified 79f90fbd

Update src/lighteval/tasks/registry.py

clefourrier committed 300 days ago

Verified 724ebf62

Update src/lighteval/tasks/registry.py

clefourrier committed 300 days ago

Verified 4b008569

Update tests/utils.py

clefourrier committed 300 days ago

Verified 8f989faa

allow old task name but logs a warning

clefourrier committed 301 days ago

a5e590c8

clefourrier committed 301 days ago

27bdf898

Merge branch 'main' into clem-add-ifbench

clefourrier committed 304 days ago

Verified 4f69c6ca

fixed last tests

clefourrier committed 304 days ago

fa9a4b8d

fix registry mockup

clefourrier committed 304 days ago

65c13414

reduces cli args redundancy (#932)

NathanHB committed 305 days ago

Verified ee31223d

fix tests part 1 - also removes fewshot truncation in the task name because it's no longer used anywhere in the code logically

clefourrier committed 305 days ago

70378150

Fixe Sampling Metrics and Evals (#938)

NathanHB committed 305 days ago

Verified 0b288e3c

clefourrier committed 306 days ago

fbebba7b

clefourrier committed 306 days ago

833e67dc

incredible how much code was just pulled from ifeval

clefourrier committed 306 days ago

bc882cdd

singleton + transformer sampling fix in config

clefourrier committed 306 days ago

0adf37e4

clefourrier committed 306 days ago

09e3f97b

better suite management: we don't load all optional deps all the time

clefourrier committed 306 days ago

197acb75

unrelated but these tasks were buggy

clefourrier committed 307 days ago

cfff0e90

clefourrier committed 307 days ago

0ecd7b89

Fix `TGI` (Text Generation Inference) Endpoint Inference and TGI JSON Grammar Generation (#502)

cpcdoy committed 308 days ago

Verified da8466bd

Parametrizing the sampling evals from the CLI (#926)

clefourrier committed 308 days ago

Verified 52d3d333

Save reasoning_content from litellm as details (#929)

muupan committed 311 days ago

Verified 50756ef2

Add Filipino Benchmark Community Tasks (#852)

ljvmiranda921 committed 311 days ago

Verified d950a1e4

Test for the xet hub team (#928)

clefourrier committed 311 days ago

Verified f586400c

fix tasks list (#906)

alielfilali01 committed 311 days ago

Verified 0da128f5

Fix #878: load tasks before models (#931)

clefourrier committed 311 days ago

Verified 222a31cd

Trackio when available (#930)

NathanHB committed 312 days ago

Verified 0da24778

Upgrade to datasets 4.0 (#924)

NathanHB committed 312 days ago

Verified e53587ef

Older