pytorch

c4bed869 - [PG Wrapper] Enhance error msg (#100213)

[PG Wrapper] Enhance error msg (#100213) Previously, the mismatch report would not give the full details of the collective running on the mismatched rank, it would look something like: ``` Detected mismatch between collectives on ranks. Rank 26 is running collective: CollectiveFingerPrint(SequenceNumber=683057617, OpType=BROADCAST, TensorShape=[1], TensorDtypes=Long, TensorDeviceTypes=TensorOptions(dtype=float (default), device=cpu, layout=Strided (default), requires_grad=false (default), pinned_memory=false (default), memory_format=(nullopt))), but Rank 1 is running collective: CollectiveFingerPrint(SequenceNumber=513876813OpType=BROADCAST). ``` i.e. Rank 1 is missing more details such as the shape, type etc. This was due to `num_tensors` field not being populated, which operator<< checks to determine whether to print additional information such as the tensor shape. Adding this field gives a better error: ``` Detected mismatch between collectives on ranks. Rank 0 is run ning collective: CollectiveFingerPrint(SequenceNumber=1564312518, OpType=ALLREDUCE , TensorShape=[20, 10], TensorDtypes=Float, TensorDeviceTypes=TensorOptions(dtype= float (default), device=cpu, layout=Strided (default), requires_grad=false (defaul t), pinned_memory=false (default), memory_format=(nullopt))), but Rank 1 is runnin g collective: CollectiveFingerPrint(SequenceNumber=1564312518, OpType=REDUCE, Tens orShape=[20, 10], TensorDtypes=Float, TensorDeviceTypes=TensorOptions(dtype=float (default), device=cpu, layout=Strided (default), requires_grad=false (default), pi nned_memory=false (default), memory_format=(nullopt))). ``` Differential Revision: [D45372325](https://our.internmc.facebook.com/intern/diff/D45372325/) Pull Request resolved: https://github.com/pytorch/pytorch/pull/100213 Approved by: https://github.com/H-Huang