transformers
[`distributed`] Divide the expert gradients once when `efsdp_size == 1`
#50292

Merged

[`distributed`] Divide the expert gradients once when `efsdp_size == 1` #50292

3outeille merged 5 commits into main from fix-ep-efsdp1-grad-division

Divide the expert gradients once when efsdp_size == 1

78f529fc

Shorten the efsdp_size == 1 comment

d9f193b4

Move the efsdp_size == 1 comment under the if

a95f97b7

3outeille requested a review from

ArthurZucker 2 days ago

Merge branch 'main' into fix-ep-efsdp1-grad-division

7f7edc0e

3outeille marked this pull request as ready for review 2 days ago

Simplify the bf16 expert grads test docstrings

8b20e7a2

ArthurZucker approved these changes on 2026-11-05

3outeille enabled auto-merge 2 days ago

3outeille merged b3187c7f into main 2 days ago

3outeille deleted the fix-ep-efsdp1-grad-division branch 2 days ago

Reviewers

ArthurZucker

Assignees

No one assigned

Labels

None yet

Milestone

No milestone