DeepSpeed
Fix ZeRO-1/2 CPU-offloaded gradient loss with multiple backward() per step
#7981

Merged

Fix ZeRO-1/2 CPU-offloaded gradient loss with multiple backward() per step #7981

delock merged 2 commits into deepspeedai:master from roycho96:fix/zero2-offload-ga1-multi-backward

Fix ZeRO-1/2 CPU-offloaded gradient loss with multiple backward() per…

70e4e69a

roycho96 requested a review from

tjruwase 52 days ago

roycho96 requested a review from

tohtana 52 days ago

roycho96 requested a review from

loadams 52 days ago

delock approved these changes on 2026-04-21

fix formatting

efd10ee3

roycho96 force pushed from 95d73e28 to efd10ee3 51 days ago

delock merged aeb10bb1 into master 50 days ago

roycho96 deleted the fix/zero2-offload-ga1-multi-backward branch 50 days ago

Reviewers

delock

tjruwase

tohtana

loadams

Assignees

No one assigned

Labels

None yet

Milestone

No milestone