Refactor commonalities between two approaches (#62624)
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/62624
Test Plan: Imported from OSS
Reviewed By: mrshenli
Differential Revision: D30058543
Pulled By: andwgu
fbshipit-source-id: 73c794062b75e011868fae264f592549eed67482
