huggingface/lighteval

Pull Requests Commits

Nathan Habib committed 1 year ago

b937183a

Add mt-bench (#75)

NathanHB committed 1 year ago

Verified af24080e

Update test workflow name to 'Tests' (#134)

Wauplin committed 1 year ago

Verified bbe3b5f8

Do not use deprecated list_files_info (#133)

Wauplin committed 1 year ago

Verified 7f25cce7

Simplified system for extended tasks (#123)

clefourrier committed 1 year ago

Verified e941a552

Adding support for Arabic benchmarks : AlGhafa benchmarking suite (#95)

alielfilali01 committed 1 year ago

Verified ef631cf5

Fix TextGenerationResponse import from hfh (#129)

Wauplin committed 1 year ago

Verified 53cc1360

Added revision parameter for Inference Endpoint deployment (#127)

shaltielshmid committed 1 year ago

Verified a82387e6

Added support for launching inference endpoint with different model dtypes (#124)

shaltielshmid committed 1 year ago

Verified 07ca0b28

Adding BBH subsets back (#126)

bilgehanertan committed 1 year ago

Verified eb0d898e

fixed the loglikelihood models, was not returning the boolean value (#119)

clefourrier committed 1 year ago

Verified 9a1697d7

Fix import typo autogptq (#116)

clefourrier committed 1 year ago

Verified 50b6ed33

Add AGIEval (#121)

clefourrier committed 1 year ago

Verified 133cf9b8

NathanHB committed 1 year ago

Verified ce6a4cc4

Small fixes to InferenceEndpointModel (#112)

shaltielshmid committed 1 year ago

Verified 491a3a76

Fix push details to hub (#98)

NathanHB committed 1 year ago

Verified b1aa6262

Ensure chat models terminate generation with EOS token (#115)

lewtun committed 1 year ago

Verified 11874009

Fix extended tasks usage in README (#108)

lewtun committed 1 year ago

Verified 6d20cb58

Adding TinyBench (#104)

clefourrier committed 1 year ago

Verified 058f3038

Reorder addition of instruction in chat template (#111)

clefourrier committed 1 year ago

Verified 1532a732

Supports extended tasks (#101)

clefourrier committed 1 year ago

Verified df21407d

Fixes input length management for generative evals (#103)

clefourrier committed 1 year ago

Verified e8e4df70

clefourrier committed 1 year ago

Verified e324a83a

Rolling management (#78)

clefourrier committed 1 year ago

Verified bca2b1df

Makes it clearer in the README that the leaderboard uses the harness (#94)

clefourrier committed 1 year ago

Verified 988959cb

Release: v0.2.0 (#77)

NathanHB committed 1 year ago

Verified 13d7946d

Change the eos condition for GSM8K (#85)

clefourrier committed 1 year ago

Verified 9b3813ff

Fix parallel data processing bug (#92)

clefourrier committed 1 year ago

Verified 3b0aa23d

add license header to src files (#89)

NathanHB committed 1 year ago

Verified 2d529ac8

Sets a max length for the MATH task (#83)

clefourrier committed 1 year ago

Verified 458d50b0

Older