PR #6024 Use reduce-scatter coalescing for FSDP

Use reduce-scatter coalescing for FSDP #6024

JackCaoG merged 3 commits into master from jeffhataws_fsdp_coaelesce

alanwaketan requested a review from

alanwaketan 2 years ago

jeffhataws force pushed from dd3bdac5 to 1bfae0e4 2 years ago

JackCaoG added backport_2.2

jeffhataws force pushed from 1bfae0e4 to 66636c8a 2 years ago

alanwaketan commented on 2023-12-14

jeffhataws force pushed from f06545da to 3dce3256 1 year ago

Separate PR for FSDP using Reduce-Scatter with bucketing/coalescing

bcbfd15b

Allgather Reduction Optimization in FSDP

aac286bf

jeffhataws force pushed from 3dce3256 to aac286bf 1 year ago

Merge branch 'master' into jeffhataws_fsdp_coaelesce

97b21f11

jeffhataws requested a review from

alanwaketan 1 year ago

alanwaketan approved these changes on 2024-08-01

JackCaoG merged 7fe070a6 into master 1 year ago

jeffhataws deleted the jeffhataws_fsdp_coaelesce branch 1 year ago

Reviewers

alanwaketan

Assignees

No one assigned

Labels

backport_2.2

Milestone

No milestone