DeepSpeed
ZeRO-3: copy offloaded master weights back to the device through pinned buffer
#8840

Merged

ZeRO-3: copy offloaded master weights back to the device through pinned buffer #8840

pengdurice merged 7 commits into deepspeedai:master from pengdurice:peng-fix-zero3-offload-pinned-copy-v2

ZeRO-3: copy offloaded master weights back to the device through pinn…

c2b85b64

Merge branch 'master' into peng-fix-zero3-offload-pinned-copy-v2

f8d55023

pengdurice marked this pull request as ready for review 18 days ago

pengdurice requested a review from

tjruwase 18 days ago

pengdurice requested a review from

tohtana 18 days ago

pengdurice requested a review from

loadams 18 days ago

delock requested a review from

delock 15 days ago

delock commented on 2026-10-23

Skip the pinned copy-back test on the CPU accelerator

d27369f5

delock approved these changes on 2026-10-23

sfc-gh-truwase commented on 2026-10-24

ZeRO-3 offload: pin the CPU master partitions and cast them on the de…

14b7942e

sfc-gh-truwase commented on 2026-10-29

Update deepspeed/runtime/zero/stage3.py

9dc2655a

ZeRO-3 offload: copy each pinned master sub-group back in one .to(dev…

1e8e7837

pengdurice force pushed from f57752ef to 1e8e7837 8 days ago

sfc-gh-truwase approved these changes on 2026-10-30

Merge branch 'master' into peng-fix-zero3-offload-pinned-copy-v2

e27ec17d

pengdurice merged c6291b7e into master 8 days ago

pengdurice deleted the peng-fix-zero3-offload-pinned-copy-v2 branch 8 days ago

Reviewers

sfc-gh-truwase

delock

tjruwase

tohtana

loadams

Assignees

No one assigned

Labels

None yet

Milestone

No milestone