DeepSpeed
edaa7221 - Fix ZeRO-1/2 with zero-sized parameters (#8280)

Commit
5 days ago
Fix ZeRO-1/2 with zero-sized parameters (#8280) ## Summary - skip zero-sized parameters before ZeRO-1/2 gradient reduction - add regression coverage for stages 1 and 2 ## Testing - `pytest -q tests/unit/runtime/zero/test_zero_empty_param.py -s` - original #8279 reproducer on A6000 with PyTorch 2.13.0+cu126 for ZeRO-1 and ZeRO-2 - `pre-commit run --files deepspeed/runtime/zero/stage_1_and_2.py tests/unit/runtime/zero/test_zero_empty_param.py` Fixes #8279 Signed-off-by: Cao Yuhang <caoyuhang@fwerkor.com>
Author
Parents
Loading