PR #1407 Big science related changes

[squash] Big science v1 (#1406)

jeffra committed 5 years ago

Merge branch 'master' into big-science-v2

jeffra committed 5 years ago

Merge branch 'master' into big-science-v2

jeffra committed 5 years ago

fix unscale_and_clip_grads mocking in moe tests

jeffra committed 5 years ago

fix for clipping mock in test_unfused_fp16_optimizer_gradnorm_for_moe

jeffra committed 5 years ago

2 workarounds for tests to work w. newer PP

jeffra committed 5 years ago

formatting

jeffra committed 5 years ago

remove incorrect set_train_batch_size from engine

jeffra committed 5 years ago

DeepSpeed Big science related changes #1407 Merged