DeepSpeed
f062a1b2 - get_grad_norm_direct: fix a case of empty norm group (#5148)

Commit
1 year ago
get_grad_norm_direct: fix a case of empty norm group (#5148) fix for [#5145 ](https://github.com/microsoft/DeepSpeed/issues/5145) empty norm group create a norm tensor with shape=[1], while other norms will be shapeless. torch.stack does not support such case. Fixing empty group norm to be shapless as well, instead of shape=[1]. --------- Co-authored-by: Lev Kurilenko <113481193+lekurile@users.noreply.github.com> Co-authored-by: Lev Kurilenko <lekurile@microsoft.com>
Author
Parents
Loading