DeepSpeed
DeepCompile ZeRO-3: robust allgather for uneven shards; fix profiling…
#7489
Merged
Merged
DeepCompile ZeRO-3: robust allgather for uneven shards; fix profiling… #7489
tohtana
merged
22 commits into
deepspeedai:master
from
juyterman1000:fix/dc-zero3-allgather-uneven-shards
eternalNight
commented
on 2025-08-18
DeepCompile ZeRO-3: robust allgather for uneven shards; fix profiling…
2e6f1b83
DeepCompile ZeRO-3: remove size-allgather; enforce uniform shard size…
24ceb0b4
Z3: allgatherParam pre-allocates padded buffer and returns a sliced v…
94e3dd6b
Z3: use .contiguous() for NCCL allgather send buffer; add comment
ffa2abaf
Merge branch 'master' into fix/dc-zero3-allgather-uneven-shards
04f9bd49
Merge branch 'master' into fix/dc-zero3-allgather-uneven-shards
80ff7353
Fix compilation error and add test for uneven shard assertion
6ccf8690
Merge branch 'master' into fix/dc-zero3-allgather-uneven-shards
04e7617a
Merge branch 'master' into fix/dc-zero3-allgather-uneven-shards
daa3e73c
Fix allgather lvalue reference error
4320c5a3
Merge branch 'master' into fix/dc-zero3-allgather-uneven-shards
0c13f9e8
Merge branch 'master' into fix/dc-zero3-allgather-uneven-shards
d0382009
Fix ZeRO-3 DeepCompile to handle padded parameters correctly - Update…
9ce693ca
Merge branch 'master' into fix/dc-zero3-allgather-uneven-shards
a01d8996
Merge branch 'fix/dc-zero3-allgather-uneven-shards' of github.com:juy…
9e6ea018
Z3 registration: compute padded-per-rank from ds_shape (total_numel) …
4f322d4d
Z3 allgatherParam: derive padded_numel via padded_per_rank from ds_shape
05a13ed5
Merge remote-tracking branch 'upstream/master' into fix/dc-zero3-allg…
c8f11bbd
Merge branch 'master' into fix/dc-zero3-allgather-uneven-shards
c8440106
Merge branch 'master' into fix/dc-zero3-allgather-uneven-shards
ea3ad681
z3: fix registration allgather output tensor shape (ProcessGroup::all…
f3f7cf5c
tohtana
approved these changes
on 2025-09-22
fix formatting
09c1be38
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub