PR #28256 Fix load balancing loss func for mixtral

Fix load balancing loss func for mixtral #28256

ArthurZucker merged 3 commits into huggingface:main from liangxuZhang:fix_load_balancing_loss_func_for_mixtral

ArthurZucker commented on 2024-01-02

ArthurZucker approved these changes on 2024-01-10

Correct the implementation of auxiliary loss of mixtrtal

64ec7cd1

correct the implementation of auxiliary loss of mixtrtal

fc6b8b09

Implement a simpler calculation method

0fe02443

liangxuZhang force pushed to 0fe02443 2 years ago

ArthurZucker merged e768616a into main 2 years ago

Reviewers

ArthurZucker

Assignees

No one assigned

Labels

None yet

Milestone

No milestone