DeepSpeed
Unmanaged gradient accumulation: ZeRO offload support
#8225

Merged

Unmanaged gradient accumulation: ZeRO offload support #8225

sfc-gh-truwase merged 6 commits into master from sfc-gh-truwase/gas_mgmt_offload

Extend unmanaged gradient accumulation to ZeRO offload

d3dcf9ad

sfc-gh-truwase requested a review from

tjruwase 16 days ago

sfc-gh-truwase requested a review from

tohtana 16 days ago

sfc-gh-truwase requested a review from

loadams 16 days ago

chatgpt-codex-connector commented on 2026-08-06

sfc-gh-truwase requested a review from

stas00 16 days ago

sfc-gh-truwase requested a review from

delock 16 days ago

Skip stale offload grads for inactive params in unmanaged finalize

e425f836

Add unmanaged ZeRO-3 NVMe offload equivalence tests

636d8f2a

Merge branch 'master' into sfc-gh-truwase/gas_mgmt_offload

925fe4c7

delock commented on 2026-08-08

delock approved these changes on 2026-08-08

delock commented on 2026-08-08

delock requested changes on 2026-08-08

Use getter/setter for ZeRO GAS boundary flag

fbed95e4

delock approved these changes on 2026-08-09

Merge branch 'master' into sfc-gh-truwase/gas_mgmt_offload

6b6a5269

sfc-gh-truwase merged cf443004 into master 13 days ago

sfc-gh-truwase deleted the sfc-gh-truwase/gas_mgmt_offload branch 13 days ago

Reviewers

delock

tjruwase

tohtana

loadams

stas00

Assignees

No one assigned

Labels

None yet

Milestone

No milestone