DeepSpeed
feat(zenflow): run the overlapped CPU optimizer in a native process
#8058
Merged
Merged
feat(zenflow): run the overlapped CPU optimizer in a native process #8058
delock
merged
17 commits into
master
from
zenflow-cpu-op
Fix ZenFlow NaN under PyTorch-style backward via backward_prologue
d52a4ecb
Add fused multi-tensor CPU Adam for ZenFlow overlap step
1d9f3cce
Let CPU Adam kernel run serially without OpenMP
790a83a8
Add ZenFlowAdam: in-process overlapped CPU Adam
1640828d
Run ZenFlow stage 1/2 overlapped optimizer in-process
d60c7779
Run ZenFlow stage 1/2 overlapped optimizer in a separate native process
40491c90
Run ZenFlow stage 3 overlapped optimizer in the native process
4164f14d
Fail fast if the ZenFlow optimizer process does not start
60181d99
Stream ZenFlow optimizer copyback in chunks to bound the GPU memory peak
59daa9e1
Remove ZenFlow's superseded in-process overlapped optimizer path
2f9590bb
Recognize ZenFlowCPUAdam as a supported ZeRO optimizer
13ce892f
chatgpt-codex-connector
commented
on 2026-06-10
Apply clang-format formatting to ZenFlowAdam definitions
1c582212
Fail loudly if the ZenFlow optimizer process dies mid-step
3a0d10ad
delock
commented
on 2026-06-11
delock
commented
on 2026-06-11
delock
commented
on 2026-06-11
delock
commented
on 2026-06-11
delock
commented
on 2026-06-12
Copy ZenFlow optimizer results back to the GPU without staging fp32
2b7f088f
Unit-test ZenFlow's zf/pt core-affinity split
b3b52df8
Remove the orphaned fused multi-tensor CPU Adam op
f935f589
Tidy ZenFlowAdam control block: drop redundant modulo, name the hp st…
667b8f23
delock
approved these changes
on 2026-07-07
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub