DeepSpeed
Harden ZeRO-1/2 offload gradient storage lifetime and stream ordering
#8632
Merged
Go
Login via GitHub
Home
Pricing
FAQ
Install
Login
via GitHub
Overview
Commits
9
Changes
View On
GitHub
Commits
Harden ZeRO-2 CPU-offload gradient storage, synchronization, and accumulation
sfc-gh-jrasley
committed
59 days ago
Simplify ZeRO-2 offload gradient safety policy inputs
jeffra
committed
58 days ago
Preserve positional compute_grad_norm compatibility
jeffra
committed
58 days ago
Simplify ZeRO-2 offload gradient protections
jeffra
committed
57 days ago
Merge branch 'master' into jeffra/913-fixes-master-sync-r2
jeffra
committed
57 days ago
Extend offload gradient protections to ZeRO-1 and keep overlap barrier
sfc-gh-jrasley
committed
56 days ago
Merge branch 'master' into jeffra/913-fixes-master-sync-r2
jeffra
committed
51 days ago
Allocate gradient-safety test tensors on the accelerator.
sfc-gh-truwase
committed
51 days ago
Merge branch 'master' into jeffra/913-fixes-master-sync-r2
sfc-gh-truwase
committed
50 days ago
Loading