DeepSpeed
377a0d15 - replace moe checkpoint dp_world_size with seq_dp_world_size (#7732)

Commit
9 days ago
replace moe checkpoint dp_world_size with seq_dp_world_size (#7732) Replace moe checkpoint dp_world_size with seq_dp_world_size to sup moe module with seq parallel. Co-authored-by: Olatunji Ruwase <tunji.ruwase@snowflake.com>
Author
Parents
Loading