NCCL Backend support for torch.bool (#41318)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/41318
Closes https://github.com/pytorch/pytorch/issues/24137.
This PR adds support for the `torch.bool` tensor type to ProcessGroupNCCL. For most types we use the existing mapping, but since `bool` is not supported as a native `ncclDataType_t`, we add the following logic:
1) Map `at::kBool` to `ncclUint8`
2) During reduction (allreduce for example), if the operation is SUM, we instead override to to a MAX, to avoid overflow issues. The rest of the operations work with no changes. In the boolean case, changing sum to max makes no correctness difference since they both function as a bitwise OR.
The reduction logic (for example for reduce/allreduce) is as follows:
sum, max = bitwise or
product, min = bitwise and
Tests are added to ensure that the reductions work as expected.
ghstack-source-id: 108315417
Test Plan: Added unittests
Reviewed By: mrshenli
Differential Revision: D22496604
fbshipit-source-id: a1a15381ec41dc59923591885d40d966886ff556
