Fix backup solution (#40515)
Summary:
These were changes that had to be made in the `release/1.6` branch in order to get backups to work.
They should be brought to the master branch.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/40515
Differential Revision: D22221308
Pulled By: seemethere
fbshipit-source-id: 24e2a0196a8e775fe324a383c8f0c681118b741b
