Multilingual Hellaswag tasks #332
hynky1999
merged
60 commits into
main
from
hellaswag_tasks
add multilignaul dynamic generative metrics
5c69eb04
Merge branch 'main' into geneartive_dynamic_metrics
39c4220d
Merge branch 'geneartive_dynamic_metrics' into config_templates
2a5cdca9
draft
2df9a082
finish multichoice config
95729ee5
Merge branch 'main' into geneartive_dynamic_metrics
3aa05796
update tokenizers + install nltk reqs
b8f90a9e
use punkt tab
f5a8717e
Update src/lighteval/utils/imports.py
227f5729
Update src/lighteval/metrics/normalizations.py
d80b3bac
fix imports
532bdadd
remove unused import
75f7ac5d
Merge branch 'main' into geneartive_dynamic_metrics
f99e330d
Merge branch 'main' into geneartive_dynamic_metrics
92daf909
Merge branch 'main' into geneartive_dynamic_metrics
f2a801d0
finish implementation of templates + move stuff around
91d9d4f7
resolve nits
9356cc62
when in rome do as romans do (handle error messages the same way)
0fbc7313
fix utils
fa1fa834
Merge branch 'geneartive_dynamic_metrics' into config_templates
db36e166
nicers tests + fix them
44aeecfc
nicer todo
2bff9633
add nice doscrings 📃
3c9eb211
add even more docstring
4216ae29
nit
d8f56b80
fix test
f26e88cb
add multilingual to dev group
111d615a
merge nli, add languagees to literals
7ca42394
translation literals
22eeddb4
add nli
7faaa8a4
add copa tasks + fix tranlation literals
865bcbc7
add hellaswag tasks
856e758f
remove custom telgu hellaswag
0c7987f2
remove hindi hellaswag
4165fc2b
add rcb + chinese nli
ba44fe9d
Merge branch 'geneartive_dynamic_metrics' into config_templates
2d092561
Merge branch 'config_templates' into multilnag_nli_tasks
7324e896
Merge branch 'multilnag_nli_tasks' into multilang_copa_task
01161b2a
Merge branch 'multilang_copa_task' into hellaswag_tasks
680124ed
clefourrier
commented
on 2024-09-27
clefourrier
commented
on 2024-09-27
clefourrier
commented
on 2024-09-27
clefourrier
commented
on 2024-09-27
clefourrier
commented
on 2024-09-27
Update src/lighteval/tasks/multilingual/tasks.py
ca865bd7
Update src/lighteval/tasks/multilingual/tasks.py
1cc1187f
Update src/lighteval/tasks/multilingual/tasks.py
d64251f1
Update src/lighteval/tasks/multilingual/tasks.py
9806fab7
Update src/lighteval/tasks/multilingual/tasks.py
35d7e6d6
Update src/lighteval/tasks/multilingual/tasks.py
e560738b
Update src/lighteval/tasks/multilingual/tasks.py
99524c5a
add two new tasks + docs
150c76fa
Merge branch 'multilnag_nli_tasks' of github.com:huggingface/lighteva…
4e6100dd
Merge remote-tracking branch 'origin/main' into multilnag_nli_tasks
7b561fe6
Merge branch 'multilnag_nli_tasks' into multilang_copa_task
f9c71346
add nice docs
b219e070
Merge branch 'multilang_copa_task' into hellaswag_tasks
8899a1cd
update hellaswag with docs
fcad0e8d
move hellaswag to lighteval suite
43809ab5
Update src/lighteval/tasks/multilingual/tasks.py
a673d655
Merge remote-tracking branch 'origin/main' into hellaswag_tasks
3c453f79
enable returning none from templates + better typing
815e8977
change unoficial hellaswag names to have community_prefix + unify hel…
eec2444f
let strip be optional in hellaswag
6284fe5d
clefourrier
approved these changes
on 2024-10-01
Merge branch 'main' into hellaswag_tasks
9bf53e93
Login to write a write a comment.
Login via GitHub