[feat] Tensor parallelism for MiniMax-H3 #14609
sayakpaul
merged
41 commits into
huggingface:main
from
JingyaHuang:add-h3-tp-support
Add tensor-parallel support for MiniMax-H3
764e7c8a
Shard MiniMax-H3's adaln_proj to fit tensor parallelism on one device
58d9b513
Build MiniMax-H3's row timestep plan on CPU, as its caller expects
c766b888
feat:combine tp+cp
41627842
Allow tensor parallelism and context parallelism in one ParallelConfig
7f649f94
feat: validate the support on TPU
673a3f10
fix: put tp first in the combined TP+CP mesh
3db653d8
fix: style
9c0f7aaf
fix: apply context parallelism on the sharded-load path
bbc55a76
revert: attention head padding
6ce34c77
revert: drop the Neuron all-gather padding and restore the TPU pre-sh…
986ea9a4
refactor: share the CPU pre-shard path between Neuron and TPU tensor …
6845f333
Merge branch 'main' into add-h3-tp-support
e8a38d37
fix: new device count API for TPU since deprecation
2590829f
fix: pad uneven all-gathers on Neuron and TPU in _all_gather_anything
ce4198ac
Merge branch 'main' into add-h3-tp-support
d2c362ed
Merge branch 'main' into add-h3-tp-support
3cd26a95
Merge branch 'main' into add-h3-tp-support
6b3b3e89
fix: fall back to one device when no accelerator is present
cd611662
Move TP+CP composition to #14725
cc851100
Merge branch 'main' into add-h3-tp-support
b1076e6f
review: use `config._device` for the Neuron TP device
8526d942
review: replace ReplicatedInputRowwiseParallel with a "rowwise_split_…
ed452ef8
review: explain when to use "rowwise_split_input"
bc35ec2d
Merge branch 'main' into add-h3-tp-support
3ea9cc59
feat:shard tensor-parallel weights locally on every device
660bb6da
Merge branch 'main' into add-h3-tp-support
82e45d41
Merge branch 'main' into add-h3-tp-support
dd54cfb8
doc: improve the explanation of rowwise_split_input
9ec46667
review: remove useless zero division prevention
d13196bb
review: simplify the docstring
ff24da56
revert: Restore standard TP sharding and isolate the Neuron workaround
63732ec3
revert: no refactoring of in-memory sharding
8e2fa1d5
Merge branch 'main' into add-h3-tp-support
669f503d
Merge branch 'main' into add-h3-tp-support
4d136666
revert: keep the in-memory TP path as on main
b680a255
style: pika
39bcfef6
Merge branch 'add-h3-tp-support' of github.com:JingyaHuang/diffusers …
8b78dfe0
style: make fix-copies
8c74bb17
Merge branch 'main' into add-h3-tp-support
3dc01948
sayakpaul
commented
on 2026-11-05
Merge branch 'main' into add-h3-tp-support
510487e1
sayakpaul
approved these changes
on 2026-11-05
Assignees
No one assigned
Labels
documentation
models
tests
size/M
modular-pipelines
hooks
Milestone
Login to write a write a comment.
Login via GitHub