Megatron-DeepSpeed
Adding language specific validation sets for Multilingual model training
#97
Merged
Merged
Adding language specific validation sets for Multilingual model training #97
TevenLeScao
merged
19 commits into
bigscience-workshop:main
from
hadyelsahar:main
adding multiple datasets to args
073c98ef
Extra_valid_dataset arg & display on Tensorboard
287f1247
code cleaning
5a64b937
code cleaning
20c48380
add run script example for multilingual validation
64476947
rename extra-validation to periodic-eval
f7714076
hadyelsahar
changed the title
hadyelsahar
changed the title
bugfix + logging amount of training data per language
1fdf4c36
Merge branch 'main' into main
51ac6022
change "periodic-eval" to "extra-eval"
da7ab8ba
elaborate tree sturcture in Multiling run script
0f7a89bc
initialize all_x_datasets to None if empty
bca5606e
clean kwargs.get
7fe7206d
add mode two of data loading
38e1fc1b
adding option2 for data loading
5d37e043
fix missing argument and range of split
64e97765
add multilingual run script
83260327
hadyelsahar
changed the title
small cleanups + fixed bugs with test iterator
4c6fa821
Fixed prefixlm
ec007069
adding variable cutoff for last epoch rather than 80%
389936ef
Reviewers
Assignees
Labels
enhancement
multilinguality
Milestone
No milestone
Login to write a write a comment.
Login via GitHub