Merge branch 'preprocess_fix' into 'main'

Commit

4 years ago

Merge branch 'preprocess_fix' into 'main' Add arg needed for tokenizer. See merge request ADLR/megatron-lm!272