DeepSpeed
Harden ZeRO-1/2 offload gradient storage lifetime and stream ordering
#8632
Merged

Harden ZeRO-1/2 offload gradient storage lifetime and stream ordering #8632

jeffra
sfc-gh-jrasley Harden ZeRO-2 CPU-offload gradient storage, synchronization, and accu…
4b9a44b1
jeffra jeffra force pushed from 5e3922e4 to 4b9a44b1 57 days ago
stas00
stas00 commented on 2026-09-23
jeffra Simplify ZeRO-2 offload gradient safety policy inputs
c49a3ce2
jeffra jeffra marked this pull request as ready for review 57 days ago
jeffra jeffra requested a review from tjruwase tjruwase 57 days ago
jeffra jeffra requested a review from tohtana tohtana 57 days ago
jeffra jeffra requested a review from loadams loadams 57 days ago
chatgpt-codex-connector
chatgpt-codex-connector commented on 2026-09-24
jeffra Preserve positional compute_grad_norm compatibility
badf669c
sfc-gh-truwase
jeffra Simplify ZeRO-2 offload gradient protections
13283f4c
jeffra Merge branch 'master' into jeffra/913-fixes-master-sync-r2
57356215
sfc-gh-truwase
sfc-gh-jrasley Extend offload gradient protections to ZeRO-1 and keep overlap barrier
81a25298
jeffra jeffra changed the title Harden ZeRO-2 CPU-offload gradient storage, synchronization, and accumulation Harden ZeRO-1/2 offload gradient storage lifetime and stream ordering 55 days ago
jeffra Merge branch 'master' into jeffra/913-fixes-master-sync-r2
097a03d0
sfc-gh-truwase
sfc-gh-truwase approved these changes on 2026-09-30
sfc-gh-truwase sfc-gh-truwase enabled auto-merge 50 days ago
sfc-gh-truwase
sfc-gh-truwase
sfc-gh-truwase Allocate gradient-safety test tensors on the accelerator.
1c50048c
sfc-gh-truwase Merge branch 'master' into jeffra/913-fixes-master-sync-r2
bc974d17
sfc-gh-truwase sfc-gh-truwase enabled auto-merge 49 days ago
sfc-gh-truwase sfc-gh-truwase merged c687df79 into master 49 days ago
sfc-gh-truwase sfc-gh-truwase deleted the jeffra/913-fixes-master-sync-r2 branch 49 days ago
tohtana tohtana added needs-gpu-test
tohtana tohtana removed needs-gpu-test

Login to write a write a comment.

Login via GitHub

Assignees
No one assigned
Labels
Milestone