Optim foreach cleanup for ASGD (#70231)
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/70231
Test Plan: Imported from OSS
Reviewed By: anjali411
Differential Revision: D33767867
Pulled By: mikaylagawarecki
fbshipit-source-id: 4406824acbb6f427d52c1ced2d8a02a98c943b86
(cherry picked from commit cbd9a4da15e0c99410a53232aa0816050097dc3e)
