pytorch

6400d27b - [Gradient Compression] Define a customized state for PowerSGD comm hook (#48348)

[Gradient Compression] Define a customized state for PowerSGD comm hook (#48348) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/48348 To support the features like error feedback, warm start, PowerSGD comm hook needs to maintain a state besides process group. Currently this state only includes a process group and a matrix approximation rank config. This diff is a pure refactoring. Plan to add more state fields later. Original PR issue: Investigate Applying PowerSGD to Communication Hook for Gradient Compression #47202 ghstack-source-id: 117305280 Test Plan: buck test mode/dev-nosan caffe2/test/distributed:c10d -- test_powerSGD_ddp_comm_hook_nccl buck test mode/dev-nosan caffe2/test/distributed:c10d -- test_powerSGD_ddp_comm_hook_nccl_grad_is_view Reviewed By: rohan-varma Differential Revision: D25137962 fbshipit-source-id: cd72b8b01e20f80a92c7577d22f2c96e9eebdc52