DeepSpeed
Tensor-Parallelism general support
#1512
Merged
Overview
Commits
17
Changes
View On GitHub

Tensor-Parallelism general support #1512

jeffra merged 17 commits into master from inference/TP-general-support
RezaYazdaniAminabadi
fixing the softmax masking when using triangular masking
f7ef4b5e
Merge branch 'master' of github.com:microsoft/DeepSpeed
dfb603fe
Merge branch 'master' of github.com:microsoft/DeepSpeed
c5ecf325
add model-parallelism support independent of kernel-injection
0d8dd450
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from awan-10 awan-10 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from cli99 cli99 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from conglongli conglongli 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from eltonzheng eltonzheng 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from jeffra jeffra 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from minjiaz minjiaz 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from niumanar niumanar 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from samyam samyam 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from ShadenSmith ShadenSmith 4 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from tjruwase tjruwase 4 years ago
remove unnecessary changes
96185233
add gpt-2 TP support without using policy
7752ec9e
tjruwase Merge branch 'master' into inference/TP-general-support
aa9eecac
fix some issues with the Tensor-Parallelism support
ebdeda90
Merge branch 'inference/TP-general-support' of github.com:microsoft/D…
ef1298b5
RezaYazdaniAminabadi Merge branch 'master' into inference/TP-general-support
edb157a6
fix the replace-module for TP-inference support
70a4d815
Merge branch 'inference/TP-general-support' of github.com:microsoft/D…
8d147518
fix formatting
0024abc0
jeffra Merge branch 'master' into inference/TP-general-support
bd3b08f0
RezaYazdaniAminabadi Merge branch 'master' into inference/TP-general-support
4092412e
RezaYazdaniAminabadi Merge branch 'master' into inference/TP-general-support
8e71030f
RezaYazdaniAminabadi Merge branch 'master' into inference/TP-general-support
b4a0ad8b
jeffra
jeffra approved these changes on 2021-11-12
jeffra jeffra merged 9ce00a21 into master 4 years ago
hyunwoongko
hyunwoongko
hyunwoongko commented on 2022-01-22
hyunwoongko
hyunwoongko commented on 2022-01-23
RezaYazdaniAminabadi

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum