DeepSpeed
sequence parallel for uneven heads
#6392
Merged
Overview
Commits
26
Changes
View On GitHub

sequence parallel for uneven heads #6392

loadams merged 26 commits into deepspeedai:master from inkcherry:uneven_sp
inkcherry
inkcherry sequence parallel for uneven heads
f5676a38
inkcherry inkcherry requested a review from tjruwase tjruwase 1 year ago
inkcherry inkcherry requested a review from awan-10 awan-10 1 year ago
inkcherry inkcherry requested a review from arashb arashb 1 year ago
inkcherry inkcherry requested a review from loadams loadams 1 year ago
inkcherry refine code
3ebb373d
inkcherry refine code
fc0ec678
delock
delock commented on 2024-08-28
delock
delock commented on 2024-08-28
inkcherry update
836a59ac
inkcherry fix all
b13a79c5
inkcherry refine code
8210320a
inkcherry test sp8 20heads
6ffb1017
inkcherry update
b3578fbb
inkcherry update comments
e1045bf2
inkcherry update ut
8d1704ed
inkcherry update
941ce30d
inkcherry fix format
9b529eb0
inkcherry fix format
dc9b90f1
inkcherry inkcherry force pushed from 1e297d3e to fc0ec678 1 year ago
inkcherry fix comments
82eefad3
inkcherry fix comments
6d94d81c
inkcherry uneven heads sp fix
87d85993
delock
delock commented on 2024-08-28
inkcherry add assert, fix format
4af5b809
inkcherry Merge branch 'master' into uneven_sp
554ff868
inkcherry replace view with reshape
66fbb7ee
delock
loadams Merge branch 'master' into uneven_sp
3913d2a3
loadams
tjruwase Merge branch 'master' into uneven_sp
5d44b4a9
tjruwase tjruwase requested a review from tohtana tohtana 1 year ago
tjruwase
tjruwase approved these changes on 2024-10-08
tjruwase
tjruwase commented on 2024-10-08
inkcherry Update deepspeed/sequence/layer.py
89cf88aa
delock Merge branch 'master' into uneven_sp
e0c0cc0e
delock Merge branch 'master' into uneven_sp
7f49a9e4
tohtana Merge branch 'master' into uneven_sp
75df1545
loadams Merge branch 'master' into uneven_sp
be805e70
loadams loadams enabled auto-merge 1 year ago
loadams loadams merged 5fb71c0a into master 1 year ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum