DeepSpeed
Fix bf16 gradient norm divergence with ZeRO stage 0
#7839

Merged

Fix bf16 gradient norm divergence with ZeRO stage 0 #7839

tohtana merged 3 commits into deepspeedai:master from tohtana:tohtana/fix-issue7837-zero0-grad-norm

Fix ZeRO-0 + bf16 broken training: disable loss scaling and fix zero_…

773c607b

add test

c5f60af0

tohtana requested a review from

tjruwase 58 days ago

tohtana requested a review from

loadams 58 days ago

sfc-gh-truwase commented on 2026-02-11

Address PR feedback for issue #7837 loss-scale refactor

459860ce

sfc-gh-truwase approved these changes on 2026-02-12

tohtana merged 1752c2ab into master 55 days ago

Reviewers

sfc-gh-truwase

tjruwase

loadams

Assignees

No one assigned

Labels

None yet

Milestone

No milestone