MoE type hints (#5043)
This PR fixes 5 pyright errors in `deepspeed`. My main goal is to fix
the type signatures of
`split_params_into_different_moe_groups_for_optimizer` since this
affects my project's linting.
I made a few other improvements along the way:
* use more descriptive variable names (`param_group` instead of `v1`,
`moe_group` instead of `v`)
* remove a few unused variables by choosing better-suited iterators like
`dict.values()` instead of `dict.items()` or `nn.Module.parameters()`
instead of `nn.Module.named_parameters()`
* fix incorrect function type signatures
* [use simple `dict()` shallow copy instead of of unnecessary for loop
excluding a key is then immediately overwritten:
](https://github.com/microsoft/DeepSpeed/compare/master...ringohoffman:moe-type-hints?expand=1#diff-cec48b3c7def770ef2d14ac7398bfbdf0f209d2558645ffd47d0028988fa66a3L134-L138)
* [ternary to reduce duplicating long
expression](https://github.com/microsoft/DeepSpeed/compare/master...ringohoffman:moe-type-hints?expand=1#diff-cec48b3c7def770ef2d14ac7398bfbdf0f209d2558645ffd47d0028988fa66a3L101-L104)
* `isinstance()` instead of `type(...) is ...`
* `typing.cast(List[nn.Parameter], param_group['params'])` as a general
pattern for improved type hinting of its elements during iteration
---------
Co-authored-by: Michael Wyatt <michaelwyatt@microsoft.com>
