DeepSpeed
Fix cudaErrorInvalidConfiguration in attn_softmax() for large values of sequence_length*num_heads
#1239

Merged

Fix cudaErrorInvalidConfiguration in attn_softmax() for large values of sequence_length*num_heads #1239

RezaYazdaniAminabadi merged 4 commits into deepspeedai:master from dfyz:master

Fix cudaErrorInvalidConfiguration in attn_softmax() for large seq_len…

bdc20422

dfyz requested a review from

awan-10 4 years ago

dfyz requested a review from

cli99 4 years ago

dfyz requested a review from

conglongli 4 years ago

dfyz requested a review from

eltonzheng 4 years ago

dfyz requested a review from

jeffra 4 years ago

dfyz requested a review from

minjiaz 4 years ago

dfyz requested a review from

niumanar 4 years ago

dfyz requested a review from

RezaYazdaniAminabadi 4 years ago

dfyz requested a review from

samyam 4 years ago

dfyz requested a review from

ShadenSmith 4 years ago

dfyz requested a review from

tjruwase 4 years ago

Merge branch 'master' into master

a0f53586

RezaYazdaniAminabadi approved these changes on 2021-07-28

Merge branch 'master' into master

23050b2b

Merge branch 'master' into master

163b3926

RezaYazdaniAminabadi merged bfe7f0db into master 4 years ago

Reviewers

RezaYazdaniAminabadi

awan-10

cli99

conglongli

eltonzheng

jeffra

minjiaz

niumanar

samyam

ShadenSmith

tjruwase

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

DeepSpeed Fix cudaErrorInvalidConfiguration in attn_softmax() for large values of sequence_length*num_heads #1239 Merged

Fix cudaErrorInvalidConfiguration in attn_softmax() for large values of sequence_length*num_heads #1239

DeepSpeed
Fix cudaErrorInvalidConfiguration in attn_softmax() for large values of sequence_length*num_heads
#1239

Merged