xla
Add reduce-scatter coalescing for FSDP/ZeRO1
#5956

Merged

Add reduce-scatter coalescing for FSDP/ZeRO1 #5956

JackCaoG merged 4 commits into master from cc_coalesce_reducescatter

JackCaoG commented on 2023-11-30

jeffhataws requested a review from

JackCaoG 2 years ago

JackCaoG added backport_2.2

JackCaoG commented on 2023-12-04

JackCaoG commented on 2023-12-05

Add reduce-scatter coalescence support for FSDP.

51e99198

jeffhataws force pushed from a4c215a6 to be0d3f72 2 years ago

jeffhataws force pushed from be0d3f72 to 51e99198 2 years ago

Fix error checking when input is list and output is specified

3d70fc09

Add reduce-scatter test with list of tensors as input/output

69968e55

jeffhataws force pushed from 4624ec52 to 69968e55 2 years ago

JackCaoG commented on 2023-12-06

For now, error when reduce-scatter coaelesce xm API is used with outp…

9baadf50

JackCaoG approved these changes on 2023-12-07

JackCaoG merged ac94781b into master 2 years ago

Reviewers

JackCaoG

Assignees

No one assigned

Labels

backport_2.2

Milestone

No milestone