Fix multi gpu test break on Windows (#58213)
Summary:
Fixes #{issue number}
Pull Request resolved: https://github.com/pytorch/pytorch/pull/58213
Reviewed By: robieta, mruberry
Differential Revision: D28405126
Pulled By: malfet
fbshipit-source-id: 48c0aa8a113c554e3a007c1900fae2ff453cf85b
