To intorduce Functional API for multi-tensor (#60735)
Summary:
In this PR we change Multi-Tensor Optimizers to Functional API.
We can see that in the file : https://github.com/pytorch/pytorch/blob/master/torch/optim/_functional.py , there has been functional API defined for most of Optimizers. However we do not have similar file / functionality for multi tensors :
https://github.com/pytorch/pytorch/tree/master/torch/optim/_multi_tensor
Therefore we are adding it in this PR here.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/60735
Reviewed By: vincentqb
Differential Revision: D29392253
Pulled By: iramazanli
fbshipit-source-id: cebc8e7b07ab11156370f5297cfb419cd9f20b46
