Megatron-DeepSpeed
5fbe1072 - Make test to work with both bf16 and fp16 to see who fails

Commit

3 years ago

Make test to work with both bf16 and fp16 to see who fails

References

#270 - Test different layer norm

#271 - Sync layer norm

#274 - Sync 4 layer norms - bf16, fp32, optimizer states on restart

Author

thomasw21

thomasw21

Parents

Loading