DeepSpeed
ZeRO-3: copy offloaded master weights back to the device through pinned buffer
#8840

Merged

Commits

ZeRO-3: copy offloaded master weights back to the device through pinned buffers

pengdurice committed 23 days ago
Merge branch 'master' into peng-fix-zero3-offload-pinned-copy-v2

pengdurice committed 18 days ago
Skip the pinned copy-back test on the CPU accelerator

pengdurice committed 15 days ago
ZeRO-3 offload: pin the CPU master partitions and cast them on the device when copying back

pengdurice committed 11 days ago
Update deepspeed/runtime/zero/stage3.py

pengdurice committed 8 days ago
ZeRO-3 offload: copy each pinned master sub-group back in one .to(device), as #5301

pengdurice committed 8 days ago
Merge branch 'master' into peng-fix-zero3-offload-pinned-copy-v2

pengdurice committed 8 days ago