Misc-multilingual tasks #339
hynky1999
merged
91 commits into
main
from
misc_tasks
add multilignaul dynamic generative metrics
5c69eb04
Merge branch 'main' into geneartive_dynamic_metrics
39c4220d
Merge branch 'geneartive_dynamic_metrics' into config_templates
2a5cdca9
draft
2df9a082
finish multichoice config
95729ee5
Merge branch 'main' into geneartive_dynamic_metrics
3aa05796
update tokenizers + install nltk reqs
b8f90a9e
use punkt tab
f5a8717e
Update src/lighteval/utils/imports.py
227f5729
Update src/lighteval/metrics/normalizations.py
d80b3bac
fix imports
532bdadd
remove unused import
75f7ac5d
Merge branch 'main' into geneartive_dynamic_metrics
f99e330d
Merge branch 'main' into geneartive_dynamic_metrics
92daf909
Merge branch 'main' into geneartive_dynamic_metrics
f2a801d0
finish implementation of templates + move stuff around
91d9d4f7
resolve nits
9356cc62
when in rome do as romans do (handle error messages the same way)
0fbc7313
fix utils
fa1fa834
Merge branch 'geneartive_dynamic_metrics' into config_templates
db36e166
nicers tests + fix them
44aeecfc
nicer todo
2bff9633
add nice doscrings 📃
3c9eb211
add even more docstring
4216ae29
nit
d8f56b80
fix test
f26e88cb
add multilingual to dev group
111d615a
merge nli, add languagees to literals
7ca42394
translation literals
22eeddb4
add nli
7faaa8a4
add copa tasks + fix tranlation literals
865bcbc7
add hellaswag tasks
856e758f
remove custom telgu hellaswag
0c7987f2
remove hindi hellaswag
4165fc2b
add rc tasks + small nits
bc669e63
add rcb + chinese nli
ba44fe9d
Merge branch 'geneartive_dynamic_metrics' into config_templates
2d092561
Merge branch 'config_templates' into multilnag_nli_tasks
7324e896
Merge branch 'multilnag_nli_tasks' into multilang_copa_task
01161b2a
Merge branch 'multilang_copa_task' into hellaswag_tasks
680124ed
Merge branch 'hellaswag_tasks' into rc_tasks
a84e3688
add mcq tasks
b10ff74c
add continuations + general qa tasks + missed tasks
9a548062
Update src/lighteval/tasks/multilingual/tasks.py
ca865bd7
Update src/lighteval/tasks/multilingual/tasks.py
1cc1187f
Update src/lighteval/tasks/multilingual/tasks.py
d64251f1
Update src/lighteval/tasks/multilingual/tasks.py
9806fab7
Update src/lighteval/tasks/multilingual/tasks.py
35d7e6d6
Update src/lighteval/tasks/multilingual/tasks.py
e560738b
Update src/lighteval/tasks/multilingual/tasks.py
99524c5a
add two new tasks + docs
150c76fa
Merge branch 'multilnag_nli_tasks' of github.com:huggingface/lighteva…
4e6100dd
Merge remote-tracking branch 'origin/main' into multilnag_nli_tasks
7b561fe6
Merge branch 'multilnag_nli_tasks' into multilang_copa_task
f9c71346
add nice docs
b219e070
Merge branch 'multilang_copa_task' into hellaswag_tasks
8899a1cd
update hellaswag with docs
fcad0e8d
move hellaswag to lighteval suite
43809ab5
add desc to tasks
a20a24e5
Update src/lighteval/tasks/multilingual/tasks.py
a673d655
Merge remote-tracking branch 'origin/main' into hellaswag_tasks
3c453f79
enable returning none from templates + better typing
815e8977
Merge branch 'hellaswag_tasks' into rc_tasks
f9801c7f
Merge branch 'rc_tasks' into multilang_mqa_tasks
843fe9ad
add nice docs
0d7b8490
fix saving of partial fcs + better tasks doc
2a8ae3dd
change unoficial hellaswag names to have community_prefix + unify hel…
eec2444f
community rename
02c8122f
let strip be optional in hellaswag
6284fe5d
nits
32c8a4e9
fix remaning tasks in mcq
8b5a0461
fixes
8ead4de0
Merge branch 'hellaswag_tasks' into rc_tasks
d7b98c0a
nits
03bd7ba4
fix up agieval + ceval
ca9d190c
add comment
e12cc3b1
Merge remote-tracking branch 'origin/main' into rc_tasks
6f3de8cc
Merge branch 'rc_tasks' into multilang_mqa_tasks
1f8c8e69
Merge branch 'multilang_mqa_tasks' into misc_tasks
70b6b93d
hellaswag fixes
d3ba842e
hellaswag hind + mlqa + hindi/swahili arc + mintaka + triviaqa french
f523bbaf
add openai mmlu, turkish mmlu, lumi and mgsm
db397b05
improve qa readibility
6244ad1c
fix norms
0f627963
Merge remote-tracking branch 'origin/main' into misc_tasks
4fcc87d7
fix few shot splits + add boolq tasks
a4d8e20a
rename boolqa to boolq
81c6a295
Merge branch 'main' into misc_tasks
1008a1a0
NathanHB
commented
on 2024-10-10
NathanHB
commented
on 2024-10-10
NathanHB
commented
on 2024-10-10
NathanHB
approved these changes
on 2024-10-10
Update src/lighteval/tasks/default_prompts.py
1a7fd132
Update src/lighteval/tasks/default_prompts.py
c299f89f
Merge branch 'main' into misc_tasks
de16271b
Login to write a write a comment.
Login via GitHub