Megatron-DeepSpeed
Adding language specific validation sets for Multilingual model training
#97

Merged

Commits

adding multiple datasets to args

hadyelsahar committed 4 years ago
Extra_valid_dataset arg & display on Tensorboard

hadyelsahar committed 4 years ago
code cleaning

hadyelsahar committed 4 years ago
code cleaning

hadyelsahar committed 4 years ago
add run script example for multilingual validation

hadyelsahar committed 4 years ago
rename extra-validation to periodic-eval

hadyelsahar committed 4 years ago
bugfix + logging amount of training data per language

TevenLeScao committed 4 years ago
Merge branch 'main' into main

TevenLeScao committed 4 years ago
change "periodic-eval" to "extra-eval"

hadyelsahar committed 4 years ago
elaborate tree sturcture in Multiling run script

hadyelsahar committed 4 years ago
initialize all_x_datasets to None if empty

hadyelsahar committed 4 years ago
clean kwargs.get

hadyelsahar committed 4 years ago
add mode two of data loading

hadyelsahar committed 4 years ago
adding option2 for data loading

hadyelsahar committed 4 years ago
fix missing argument and range of split

hadyelsahar committed 4 years ago
add multilingual run script

hadyelsahar committed 4 years ago
small cleanups + fixed bugs with test iterator

TevenLeScao committed 4 years ago
Fixed prefixlm

TevenLeScao committed 4 years ago
adding variable cutoff for last epoch rather than 80%

TevenLeScao committed 4 years ago