Add more tests for mt optimizers (#45475)
Summary:
Add more test cases for mt optimizers and fix Adam/AdamW
Pull Request resolved: https://github.com/pytorch/pytorch/pull/45475
Reviewed By: soumith
Differential Revision: D23982727
Pulled By: izdeby
fbshipit-source-id: 4b24d37bd52a2fa3719d3e3a5dcf3b96990b0f5b
