EMA op (#50393)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/50393
Exponential Moving Average
Usage:
add ema_options in adagrad optimizer. For details, plz refer to the test workflow setting.
if ema_end == -1, it means ema will never end.
Test Plan:
buck test caffe2/caffe2/fb/optimizers:ema_op_optimizer_test
buck test caffe2/caffe2/fb/optimizers:ema_op_test
f240459719
Differential Revision: D25416056
fbshipit-source-id: a25e676a364969e3be2bc47750011c812fc3a62f
