DeepSpeed
4524b8db - Fix for Zero3 when MP>1 and at least one batch param undefined (#2289)

Commit

2 years ago

Fix for Zero3 when MP>1 and at least one batch param undefined (#2289) Co-authored-by: anthony.301 <anthony.301@mri.cluster> Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

References

#2289 - Fix for Zero3 when MP>1

Author

Quentin-Anthony

Parents

7e085b62

Files1

deepspeed/runtime/zero
- stage3.py

DeepSpeed 4524b8db - Fix for Zero3 when MP>1 and at least one batch param undefined (#2289)

DeepSpeed
4524b8db - Fix for Zero3 when MP>1 and at least one batch param undefined (#2289)