Handle MoE models with DeepSpeed (#2662)
* Handle MoE models with DeepSpeed
* Update launch.py
* Update test_deepspeed.py
* Update test_deepspeed.py
* Update src/accelerate/utils/dataclasses.py
Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>
* address comments
* Update deepspeed.md
---------
Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>
