Megatron-DeepSpeed
5d3150c7 - [WIP] dealing with multi-process noise (#193)

Commit
4 years ago
[WIP] dealing with multi-process noise (#193) * deal with tokenizer logs * Trigger CI * fused_kernels.load noise * control ds replica noise * control ds replica noise * ds info log on rank0 only * only on rank 0 * only if --deepspeed * fix * replica logging for transformers
Author
Parents
Loading