huggingface/lighteval

Pull Requests Commits

clefourrier committed 1 year ago

983e8d1e

Merge branch 'clem_customizable_metrics' of github.com:huggingface/lighteval into clem_customizable_metrics

clefourrier committed 1 year ago

ddc741f8

more info about new tasks

clefourrier committed 1 year ago

8b966111

Merge branch 'main' into clem_customizable_metrics

clefourrier committed 1 year ago

Verified 645c82a3

Just adding the custom metrics system (#65)

clefourrier committed 1 year ago

Verified 3785d852

Merge branch 'main' into clem_customizable_metrics

lewtun committed 1 year ago

Verified 9da1282c

Fixes chat template application to choices (#67)

clefourrier committed 1 year ago

Verified 49074998

clefourrier committed 1 year ago

7c8ef073

Remove the eos token override in the Default Config Task (#54)

clefourrier committed 1 year ago

Verified 449817f6

thomwolf committed 1 year ago

Verified 589e6b0d

Update leaderboard task set (#60)

lewtun committed 1 year ago

Verified 6a3e3b92

Tweak installation / usage sections of README (#55)

lewtun committed 1 year ago

Verified 480d85ef

Adding support for Arabic benchmarks : AceGPT benchmarking suite (#44)

alielfilali01 committed 1 year ago

Verified 090101f1

Merge branch 'clem_customizable_metrics' of github.com:huggingface/lighteval into clem_customizable_metrics

clefourrier committed 1 year ago

75e29ab8

changed eos token

clefourrier committed 1 year ago

885541b1

Merge branch 'main' into clem_customizable_metrics

NathanHB committed 1 year ago

Verified 92f72aa5

Update tasks_examples/custom_tasks_with_custom_metrics/ifeval/ifeval.py

clefourrier committed 1 year ago

Verified 53c11587

New mechanism for evaluation contributions (#47)

clefourrier committed 1 year ago

Verified 92e9b505

fix eos token + eval script

clefourrier committed 1 year ago

1ba178f1

clefourrier committed 1 year ago

0e30b215

custom metrics working! need to update the readme

clefourrier committed 1 year ago

2fdceb82

Merge branch 'main' into clem_customizable_metrics

clefourrier committed 1 year ago

96aa81b3

clefourrier committed 1 year ago

Verified 831ad47b

Improve the current chat template system (#38)

clefourrier committed 1 year ago

Verified 81fc8fda

bump transformers to 4.38 (#46)

NathanHB committed 1 year ago

Verified fb57ffc6

Add an automatic system to compute average for tasks with subtasks

clefourrier committed 1 year ago

Verified 62abc78c

Update README.md (#37)

clefourrier committed 1 year ago

Verified 77c20164

init ifeval, now need to add loading custom metric system

clefourrier committed 1 year ago

89e7fda5

tolgacangoz committed 1 year ago

Verified c62e48ef

Patch fix to match with config update/simplification in nanotron (#35)

thomwolf committed 1 year ago

Verified 69897372

Older