Optim foreach cleanup for SGD (#70481)
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/70481
Test Plan: Imported from OSS
Reviewed By: anjali411
Differential Revision: D33767868
Pulled By: mikaylagawarecki
fbshipit-source-id: 89b9227a4ddf99602855973cbc343c58ae3d5328
(cherry picked from commit ffea8ddcfd39f3f33e18d1c7b2b903d5464d5eb9)
