ZeRO-3 partial offload: fix LR schedules and checkpoints of the device sub-groups #8973
ZeRO-3 partial offload: follow hyperparameter schedules on the device…
afded177
ZeRO-3 partial offload: load the device sub-groups' moments in the pa…
d53ffb6a
Merge branch 'master' into xinyu/zero3-partial-offload-optimizer
bf034e0e
ZeRO-3 partial offload: reject bias_correction=False restored by a ch…
3ee8c678
Merge branch 'master' into xinyu/zero3-partial-offload-optimizer
8835ce4e
ZeRO-3 partial offload tests: skip the bias_correction checkpoint tes…
d146f791
Merge branch 'master' into xinyu/zero3-partial-offload-optimizer
78272d3e
xylian86
merged
25eaac4f
into master 1 day ago
xylian86
deleted the xinyu/zero3-partial-offload-optimizer branch 1 day ago
Assignees
No one assigned
Login to write a write a comment.
Login via GitHub