DeepSpeed
8ac42ed7 - Fix redundant seq data parallel grp argument in Z3/MiCS (#5352)

Commit
1 year ago
Fix redundant seq data parallel grp argument in Z3/MiCS (#5352) Deprecate redundant sequence_data_parallel_group argument. Users/client code will control across which process group Z3 parameters will be partitioned from one of [None, data_parallel_group, sequence_data_parallel]. --------- Co-authored-by: Masahiro Tanaka <81312776+tohtana@users.noreply.github.com> Co-authored-by: Logan Adams <114770087+loadams@users.noreply.github.com>
Author
Parents
Loading