DeepSpeed

1b08325d - [TiledMLP] moe support (#7622)

[TiledMLP] moe support (#7622) MoE routers seem to drop the `bs` dimension in `x` so the `[bs, seqlen, hidden_size]` is no longer expected. support that use-case. Signed-off-by: Stas Bekman <stas@stason.org>