Use leaky singletons for torch.distributed. (#32923)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/32923
As per
https://isocpp.org/wiki/faq/ctors#construct-on-first-use-v2 and
https://isocpp.org/wiki/faq/ctors#static-init-order-on-first-use-members, we
should be using leaky singletons to avoid static initialization order problem.
Closes https://github.com/pytorch/pytorch/issues/27412
ghstack-source-id: 97601384
Test Plan: waitforbuildbot
Differential Revision: D19688986
fbshipit-source-id: 8c1935fb7da8a7116dbca55eb43dc04bc02695ac
