Maintain submodule references during subgraph rewriting (#55463)
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/55463
Test Plan: Imported from OSS
Reviewed By: jamesr66a
Differential Revision: D27621650
Pulled By: ansley
fbshipit-source-id: e3558c64cdc2c1d846355fa58307a18c0714874b