DeepSpeed
Use expert gradients, not expert weights, in the FP16_Optimizer MoE grad norm
#8635
Merged
Go
Login via GitHub
Home
Pricing
FAQ
Install
Login
via GitHub
Overview
Commits
1
Changes
View On
GitHub
Use expert gradients, not expert weights, in the FP16_Optimizer MoE grad norm
#8635
tohtana
merged 1 commit into
deepspeedai:master
from
yh0903:yh0903/fix-moe-grad-norm-uses-weights
Fix MoE fp16 grad norm to use expert grads
43cb38c8
yh0903
force pushed
from
ebe360e6
to
43cb38c8
57 days ago
yh0903
marked this pull request as ready for review
56 days ago
yh0903
requested a review
from
tjruwase
56 days ago
yh0903
requested a review
from
tohtana
56 days ago
yh0903
requested a review
from
loadams
56 days ago
pengdurice
added
will-review
tohtana
approved these changes on 2026-09-29
tohtana
enabled auto-merge
52 days ago
tohtana
merged
1b60c549
into master
52 days ago
gpu-test-labeler
added
skip-gpu-test
Login to write a write a comment.
Login via GitHub
Reviewers
tohtana
tjruwase
loadams
Assignees
No one assigned
Labels
will-review
skip-gpu-test
Milestone
No milestone
Login to write a write a comment.
Login via GitHub