DeepSpeed
93524c89 - Fix Zero-3 static scale assertion in fp16 test (#7866)

Commit
6 days ago
Fix Zero-3 static scale assertion in fp16 test (#7866) PR #7839 introduced a regression by changing `TestZeroStaticScale` from `assert optim.dynamic_loss_scale == False` to `assert optim.loss_scale_config.dynamic_loss_scale == False`. `loss_scale_config` is not part of the ZeRO optimizer (only non-ZeRO optimizer have it), while this test runs with ZeRO optimizers. With this fix, `TestZeroStaticScale` now passes for stages 1/2/3. Signed-off-by: Masahiro Tanaka <mtanaka@anyscale.com>
Author
Parents
Loading