Enable ZeRO3 allgather for multiple dtypes (#4647)
This PR addresses an error reported in #4295.
When parameters in multiple data types are given, DeepSpeed performs
allgather for each data type.
---------
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Logan Adams <114770087+loadams@users.noreply.github.com>
