Harden ZeRO-1/2 offload gradient storage lifetime and stream ordering #8632
Harden ZeRO-2 CPU-offload gradient storage, synchronization, and accu…
4b9a44b1
jeffra
force pushed
from
5e3922e4
to
4b9a44b1
57 days ago
stas00
commented
on 2026-09-23
Simplify ZeRO-2 offload gradient safety policy inputs
c49a3ce2
jeffra
marked this pull request as ready for review 57 days ago
Preserve positional compute_grad_norm compatibility
badf669c
Simplify ZeRO-2 offload gradient protections
13283f4c
Merge branch 'master' into jeffra/913-fixes-master-sync-r2
57356215
Extend offload gradient protections to ZeRO-1 and keep overlap barrier
81a25298
jeffra
changed the title Harden ZeRO-2 CPU-offload gradient storage, synchronization, and accumulation Harden ZeRO-1/2 offload gradient storage lifetime and stream ordering 55 days ago
Merge branch 'master' into jeffra/913-fixes-master-sync-r2
097a03d0
Allocate gradient-safety test tensors on the accelerator.
1c50048c
Merge branch 'master' into jeffra/913-fixes-master-sync-r2
bc974d17
sfc-gh-truwase
deleted the jeffra/913-fixes-master-sync-r2 branch 49 days ago
Assignees
No one assigned
Login to write a write a comment.
Login via GitHub