Momentum setting in SyncBatchNorm forward (inference) pass. (#24995)
Summary:
This is a fix for a potential ONNX export issue with SyncBatchNorm where irrespective of the value of momentum, the value for momentum in ONNX BN node is always 0. The details are captured in https://github.com/pytorch/pytorch/issues/18525.
The fix in this PR for `SyncBatchNorm` is very similar to the fix that went in https://github.com/pytorch/pytorch/pull/18764 for `BatchNorm` (I think this site was just missed).
Please note that there are no ONNX test points added for this, because SyncBatchNorm works exclusively with tensors on GPU and the ONNX test passes are CPU only. If there's a way to add a test point, please let me know.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/24995
Differential Revision: D17085570
Pulled By: dzhulgakov
fbshipit-source-id: 162d428673c269efca4360fb103854b7319ec204
