PR #50313 Normalize moe loss

Normalize moe loss #50313

SunMarc wants to merge 9 commits into main from moe-aux-loss-scaling

moe aux scaling

762b040f

c663d27a

SunMarc requested a review from

vasqu 5 days ago

SunMarc requested a review from

qgallouedec 5 days ago

Merge main into moe-aux-loss-scaling

3c054d31

Rename the MoE aux loss test and simplify its docstring

47afc16a

Test the MoE aux loss with and without num_items_in_batch, drop the p…

544f3c27

vasqu commented on 2026-11-04

loss moe normalization

438dd074

SunMarc changed the title ~~Fix Moe aux loss scaling~~ Normalize moe loss 4 days ago

SunMarc commented on 2026-11-04

SunMarc requested a review from

vasqu 4 days ago

Merge main into moe-aux-loss-scaling

a0d3e2e6

Fix

7826cc76

Merge main into moe-aux-loss-scaling

ede58608

SunMarc requested a review from

ArthurZucker 2 days ago

qgallouedec approved these changes on 2026-11-07

Reviewers

qgallouedec

vasqu

ArthurZucker

Assignees

No one assigned

Labels

None yet

Milestone

No milestone