transformers
Fix DBRX MoE hidden size and expert GLU transposes
#47671

Merged

Fix DBRX MoE hidden size and expert GLU transposes #47671

vasqu merged 6 commits into huggingface:main from kaixuanliu:fix-dbrx-moe-dims

Fix DBRX MoE hidden size and expert GLU transposes

262a8bee

kaixuanliu marked this pull request as ready for review 6 days ago

github-actions requested a review from

ArthurZucker 6 days ago

github-actions requested a review from

ydshieh 6 days ago

vasqu approved these changes on 2026-08-03

update

e6c87da1

propogate self.d_model to ffn_config.hidden_size

e8a1ff9a

update

116c76e0

update

77dead05

update comment

bc2e0ac6

vasqu approved these changes on 2026-08-04

vasqu enabled auto-merge 4 days ago

vasqu merged 2ce15881 into main 4 days ago

kaixuanliu deleted the fix-dbrx-moe-dims branch 3 days ago

Reviewers

vasqu

ArthurZucker

ydshieh

Assignees

No one assigned

Labels

None yet

Milestone

No milestone