Add back the owning_module fix (#65159)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/65159
This was a legit fix originally introduced in D30905949 (https://github.com/pytorch/pytorch/commit/446d95a7f64cb464d28d27c4c87c48900a9fde79). But we hesitated and removed it for some reason. Putting it back.
Reviewed By: 842974287
Differential Revision: D30996277
fbshipit-source-id: 3f5eede11dba2072e7cd5ae6ca7ac81d55fb75fa
