DeepSpeed
Keep fp32 masters on ZeRO-1 and ZeRO-2 when a resume skips optimizer state
#8891

Merged

Keep fp32 masters on ZeRO-1 and ZeRO-2 when a resume skips optimizer state #8891

sfc-gh-truwase merged 7 commits into deepspeedai:master from sfc-gh-truwase:load-fp32-masters-without-optimizer-state

sfc-gh-truwase requested a review from

tjruwase 9 days ago

sfc-gh-truwase requested a review from

tohtana 9 days ago

sfc-gh-truwase requested a review from

loadams 9 days ago

chatgpt-codex-connector commented on 2026-10-22

Keep fp32 masters on ZeRO-1 and ZeRO-2 when a resume skips optimizer …

f9958d4d

sfc-gh-truwase force pushed from 945aa1c8 to f9958d4d 9 days ago

sfc-gh-truwase requested a review from

jeffra 9 days ago

sfc-gh-truwase changed the title ~~Keep fp32 masters when a resume skips optimizer state~~ Keep fp32 masters on ZeRO-1 and ZeRO-2 when a resume skips optimizer state 9 days ago

Merge master into the ZeRO-1/2 fp32 master branch.

b3728af3

sfc-gh-truwase requested a review from

stas00 9 days ago

stas00 commented on 2026-10-22

Clarify FP32 checkpoint loading paths

e489f80b

Keep saved fp32 masters on every full ZeRO load.

a7ce02d9

Fail a universal load that is missing its fp32 master.

2f56cafc

Skip DistributedFixture tests when the world size is unavailable.

894a2b72

stas00 approved these changes on 2026-10-26

Merge branch 'master' into load-fp32-masters-without-optimizer-state

f02e7675

sfc-gh-truwase merged 2dd22dc1 into master 4 days ago

sfc-gh-truwase deleted the load-fp32-masters-without-optimizer-state branch 4 days ago

Reviewers

stas00

chatgpt-codex-connector

tjruwase

tohtana

loadams

jeffra

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

DeepSpeed Keep fp32 masters on ZeRO-1 and ZeRO-2 when a resume skips optimizer state #8891 Merged

Keep fp32 masters on ZeRO-1 and ZeRO-2 when a resume skips optimizer state #8891

DeepSpeed
Keep fp32 masters on ZeRO-1 and ZeRO-2 when a resume skips optimizer state
#8891

Merged