Add managed_gradient_accumulation for ZeRO stage 0/1 #8184
tohtana
merged
12 commits into
master
from
sfc-gh-truwase/gas_mgmt_zero01
Add managed_gradient_accumulation for ZeRO stage 0/1
00800f7a
stas00
commented
on 2026-07-27
Guard unmanaged gradient accumulation against unsupported features
e7592ea5
Doc gas mgmt
dbe79089
Merge branch 'master' into sfc-gh-truwase/gas_mgmt_zero01
4c6ae671
Docs: mark unmanaged ZeRO-2/3 and offload as not yet available
c6ba3415
Docs: make unmanaged gradient accumulation example use scale_wrt_gas=…
dec2b145
stas00
commented
on 2026-07-29
Docs: make loss averaging explicit in unmanaged accumulation example
ab6e8ac7
Address tohtana review: reject set_gradient_accumulation_boundary/off…
bee8ed56
Tersify comments in unmanaged gradient accumulation tests
0191153d
Reject ZeRO overlap_comm in unmanaged gradient accumulation
6b3d3564
Track global_samples by actual micro-batch count in unmanaged mode
43070642
Merge branch 'master' into sfc-gh-truwase/gas_mgmt_zero01
77fbb461
tohtana
approved these changes
on 2026-08-02
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub