DeepSpeed
[AutoTP] Make AutoTP work when num_heads not divisible by number of workers
#4011
Merged
Merged
[AutoTP] Make AutoTP work when num_heads not divisible by number of workers #4011
tjruwase
merged
55 commits into
deepspeedai:master
from
delock:gma/uneven_heads
allow number of heads not divisible by number of ranks
0706acd7
get num_heads from model config, more robust
0bf785f8
simplify logic where num_head itself is sharded
72b9e1ad
name tweaks
5ed9a566
make code more robust where num_attention_heads may not be defined in…
73f499d9
Merge branch 'master' into gma/uneven_heads
48322c7b
Merge branch 'master' into gma/uneven_heads
f14e2909
mrwyattii
requested changes
on 2023-07-24
Merge branch 'master' into gma/uneven_heads
b62317cb
support num_key_value_heads < num_attention_heads which is used by ll…
12c06286
add test for 5 ranks
8f23d9bf
change odd rank # to 3 to avoid test skip
9c53bd74
Merge branch 'master' into gma/uneven_heads
413224b2
Merge branch 'master' into gma/uneven_heads
78d6667a
add get_shard_size function
27fde308
modify sharding mechanism according to latest auto TP
8e1fd277
Merge branch 'master' into gma/uneven_heads
9a6bc12f
fix accuracy issue
2dac94f3
Merge branch 'master' into gma/uneven_heads
885f6a34
Merge branch 'master' into gma/uneven_heads
7ffd8112
Merge branch 'master' into gma/uneven_heads
40659baf
fix format
71f9f401
skip tests with fusedqkv
db9db6bf
Merge branch 'master' into gma/uneven_heads
72531c06
remove skip of fusedqkv tests
9d5eae34
skip test fusedqkv with odd number of ranks
25e656db
support model with n_heads in model_config
7f6d7f6d
Merge branch 'master' into gma/uneven_heads
e3a5b771
Merge branch 'master' into gma/uneven_heads
c9ec881a
fix TestInjectionPolicy::test[fp32-t5]
f5be2571
fix uneven_heads on some fusedqkv types (#12)
b6710401
dc3671
commented
on 2023-08-30
better fix when activation size cannot be divided by number of heads
d59ff222
Merge branch 'master' into gma/uneven_heads_rebase
6c3c841d
molly-smith
approved these changes
on 2023-09-01
Merge branch 'master' into gma/uneven_heads
58e8b24a
move tp_shard.py under module_inject
4c6b7fa2
Merge branch 'master' into gma/uneven_heads
18e1c5d5
Add get_num_kv_heads in tp_shard.py
8ef01e28
Merge branch 'master' into gma/uneven_heads
9a61fc26
Merge branch 'master' into gma/uneven_heads
74870db3
Merge branch 'master' into gma/uneven_heads
115cc208
Refine according to comments
0781c41e
disabled auto-merge 2 years ago
Head branch was pushed to by a user without write access
Head branch was pushed to by a user without write access
mrwyattii
commented
on 2023-09-14
remove old comment
194337fe
Merge branch 'master' into gma/uneven_heads
47d84caf
Merge branch 'master' into gma/uneven_heads
369eb3e1
fix bug in getting num_kv_heads
567fb9ae
Merge branch 'master' into gma/uneven_heads
47c83cac
Merge branch 'master' into gma/uneven_heads
d194ab07
mrwyattii
approved these changes
on 2023-10-02
Merge branch 'master' into gma/uneven_heads
6db5ddd0
disabled auto-merge 2 years ago
Head branch was pushed to by a user without write access
Head branch was pushed to by a user without write access
Merge branch 'up-master' into gma/uneven_heads
698b62a8
support uneven sharding of lm_head tensor parallel
d75149fb
Merge branch 'master' into gma/uneven_heads
248532db
Merge branch 'master' into gma/uneven_heads
a9056fd5
Merge branch 'master' into gma/uneven_heads
81bd29f2
Merge branch 'master' into gma/uneven_heads
693a9fea
Merge branch 'master' into gma/uneven_heads
4c45a5b7
Merge branch 'master' into gma/uneven_heads
a7513e17
Assignees
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub