DeepSpeed
ZeRO-3: reuse the weight partition as the master copy when it already has the master dtype
#8842

Merged

ZeRO-3: reuse the weight partition as the master copy when it already has the master dtype #8842

pengdurice merged 6 commits into deepspeedai:master from pengdurice:peng-fix-zero3-master-copy-device-v1

ZeRO-3: reuse the weight partition as the master copy when it already…

1409708d

0z5a commented on 2026-10-18

ZeRO-3: keep aliased master partitions with the weights in offload_st…

8045140f

Merge branch 'master' into peng-fix-zero3-master-copy-device-v1

fb392f3b

Merge branch 'master' into peng-fix-zero3-master-copy-device-v1

bd89a532

pengdurice marked this pull request as ready for review 17 days ago

pengdurice requested a review from

tjruwase 17 days ago

pengdurice requested a review from

tohtana 17 days ago

pengdurice requested a review from

loadams 17 days ago

chatgpt-codex-connector commented on 2026-10-25

0z5a commented on 2026-10-31

Merge branch 'master' into peng-fix-zero3-master-copy-device-v1

8f4af26f

ZeRO-3: warn when offloading hp_params leaves aliased sub-groups on t…

0831d18b

tohtana approved these changes on 2026-11-05

pengdurice merged ed827399 into master 6 days ago

pengdurice deleted the peng-fix-zero3-master-copy-device-v1 branch 6 days ago

tohtana added needs-gpu-test

Reviewers

tohtana

0z5a

tjruwase

loadams

Assignees

No one assigned

Labels

needs-gpu-test

Milestone

No milestone