huggingface/transformers

Pull Requests Commits

Docs: shorter token dispatch overview

qgallouedec committed 1 hour ago

4a9e39d2

Keep the base install_forward signature on the dispatch experts style

qgallouedec committed 2 hours ago

caee41dd

qgallouedec committed 2 hours ago

7e750138

Drop the FSDP plan skip from the mixin tests: every config carries a default plan

qgallouedec committed 2 hours ago

bedee949

Fix the _has_fsdp_plan docstring

qgallouedec committed 2 hours ago

ccaa0578

Dispatch: skip the mixin tests for models without an FSDP plan, drop the experts implementation check

qgallouedec committed 2 hours ago

3204fc19

Dispatch: drop the router style, reuse the named mesh and is_dtensor, check the batches at init

qgallouedec committed 2 hours ago

ceb1a5cf

Dispatch: one host sync per layer, comments say what the code does

qgallouedec committed 3 hours ago

df8d664a

Run the local experts through the module's own forward

qgallouedec committed 3 hours ago

6c163d85

Run the local experts directly: the grouped forward's masking and combine passes cost 15-25% of throughput

qgallouedec committed 3 hours ago

a117992f

Seed the dispatch sampler with data_seed; require the grouped_mm experts implementation

qgallouedec committed 3 hours ago

80ecf2b7

Rewrite the expert parallel plan the model actually reads for the dispatch styles

qgallouedec committed 3 hours ago

a96d5ddd

Select the dispatch styles in the mixin, guard the torch version and dispatch_batches

qgallouedec committed 3 hours ago

891bf857

Dispatch as plan-level styles; one notion of the ranks sharing a batch in the Trainer

qgallouedec committed 3 hours ago

56925425

Simplify the dispatch: torch's differentiable all-to-all, the grouped forward for the local experts, ids from the count exchange

qgallouedec committed 3 hours ago

165f6578

Split the batches across every rank with a DistributedSampler under token dispatch

qgallouedec committed 4 hours ago

a4882e98

Combine the dispatched expert outputs in the routing weights' precision, as the masked path does

qgallouedec committed 4 hours ago

992d9b73

Make the dispatch handling of the experts explicit in apply_fully_sharded_data_parallelism

qgallouedec committed 4 hours ago

b6e282c3

Expert-parallel token dispatch: each rank trains on its own part of the batch

qgallouedec committed 4 hours ago

e95504b5

Cosmetic: docstring layout, comment placement, doc link

qgallouedec committed 5 hours ago

393af534

Refresh the throughput table

qgallouedec committed 8 hours ago

ef412569

Test the 2-D mesh with a batch split across the fsdp ranks and compare gradient norms

qgallouedec committed 8 hours ago

508996b6

Give each device mesh its own optimizer param group instead of stepping per parameter

qgallouedec committed 8 hours ago

c18e5d99

Address review: drop the dead expert-gradient sync, torch-native mixed-mesh norm, single collective save

qgallouedec committed 8 hours ago

ad5df191

Document combining expert parallelism with FSDP2

qgallouedec committed 9 hours ago

bba3125e

Gather the sharded state dict through the model's own save helpers

qgallouedec committed 9 hours ago

71fcd09a

Enable FSDP2 + expert parallelism via a 2-D (fsdp, tp) device mesh

qgallouedec committed 9 hours ago

e3a43b26

Fix expert parallelism through Trainer (#48208)

qgallouedec committed 10 hours ago

Verified caf370a5

Gate the router-score backward all-reduce on grad mode; drop the post-mask superseded by the per-mm masks

qgallouedec committed 10 hours ago

Verified e0b2e716

qgallouedec committed 10 hours ago

Verified c21e8e72

Older