Use group.WORLD appropriately in process group initialization. (#48767)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/48767
As part of investigating
https://github.com/pytorch/pytorch/issues/48464, I realized some weird
inconsistency in how we use `_default_pg` and `group.WORLD`. `group.WORLD`
apparently was an `object()` and never changed despite `_default_pg` changing.
In this sense, `group.WORLD` was being used a constant to refer to the default
pg, but wasn't of type PG at all. In fact the passed in group is also compared
via `==` to `group.WORLD` in many places, and it just worked since the default
argument was `group.WORLD`.
To clean this up, I got rid of `_default_pg` completely and instead used
`group.WORLD` as the default pg throughout the codebase. This also fixes the
documentation issues mentioned in
https://github.com/pytorch/pytorch/issues/48464.
#Closes: https://github.com/pytorch/pytorch/issues/48464
ghstack-source-id: 118459779
Test Plan: waitforbuildbot
Reviewed By: rohan-varma
Differential Revision: D25292893
fbshipit-source-id: 9a1703c71610aee2591683ab60b010332e05e412
