DeepSpeed
ZeRO-3 partial offload: fix LR schedules and checkpoints of the device sub-groups
#8973
Merged

ZeRO-3 partial offload: fix LR schedules and checkpoints of the device sub-groups #8973

xylian86
xylian86 ZeRO-3 partial offload: follow hyperparameter schedules on the device…
afded177
xylian86 xylian86 requested a review from tjruwase tjruwase 21 days ago
xylian86 xylian86 requested a review from tohtana tohtana 21 days ago
xylian86 xylian86 requested a review from loadams loadams 21 days ago
chatgpt-codex-connector
chatgpt-codex-connector
chatgpt-codex-connector commented on 2026-11-01
xylian86 ZeRO-3 partial offload: load the device sub-groups' moments in the pa…
d53ffb6a
sfc-gh-truwase
sfc-gh-truwase approved these changes on 2026-11-09
sfc-gh-truwase
sfc-gh-truwase approved these changes on 2026-11-09
tohtana tohtana added needs-gpu-test
tohtana tohtana removed needs-gpu-test
xylian86 Merge branch 'master' into xinyu/zero3-partial-offload-optimizer
bf034e0e
xylian86 ZeRO-3 partial offload: reject bias_correction=False restored by a ch…
3ee8c678
xylian86 xylian86 enabled auto-merge 2 days ago
xylian86 Merge branch 'master' into xinyu/zero3-partial-offload-optimizer
8835ce4e
xylian86 ZeRO-3 partial offload tests: skip the bias_correction checkpoint tes…
d146f791
xylian86 Merge branch 'master' into xinyu/zero3-partial-offload-optimizer
78272d3e
xylian86 xylian86 merged 25eaac4f into master 1 day ago
xylian86 xylian86 deleted the xinyu/zero3-partial-offload-optimizer branch 1 day ago

Login to write a write a comment.

Login via GitHub

Assignees
No one assigned
Labels
Milestone