DeepSpeed
Harden ZeRO-1/2 offload gradient storage lifetime and stream ordering
#8632
Merged

Commits
  • Harden ZeRO-2 CPU-offload gradient storage, synchronization, and accumulation
    sfc-gh-jrasley committed 59 days ago
  • Simplify ZeRO-2 offload gradient safety policy inputs
    jeffra committed 58 days ago
  • Preserve positional compute_grad_norm compatibility
    jeffra committed 58 days ago
  • Simplify ZeRO-2 offload gradient protections
    jeffra committed 57 days ago
  • Merge branch 'master' into jeffra/913-fixes-master-sync-r2
    jeffra committed 57 days ago
  • Extend offload gradient protections to ZeRO-1 and keep overlap barrier
    sfc-gh-jrasley committed 56 days ago
  • Merge branch 'master' into jeffra/913-fixes-master-sync-r2
    jeffra committed 51 days ago
  • Allocate gradient-safety test tensors on the accelerator.
    sfc-gh-truwase committed 51 days ago
  • Merge branch 'master' into jeffra/913-fixes-master-sync-r2
    sfc-gh-truwase committed 50 days ago
Loading