DeepSpeed
Transformer-kernel - supporting any arbitrary sequence-length
#587
Merged
Overview
Commits
9
Changes
View On GitHub

Transformer-kernel - supporting any arbitrary sequence-length #587

jeffra merged 9 commits into master from transformer/support-arbitrary-seqlen
RezaYazdaniAminabadi
Transformer-kernel - supporting any arbitrary sequence-length
eb4700b1
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from arashashari arashashari 5 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from awan-10 awan-10 5 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from cli99 cli99 5 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from conglongli conglongli 5 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from eltonzheng eltonzheng 5 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from jeffra jeffra 5 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from minjiaz minjiaz 5 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from niumanar niumanar 5 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from samyam samyam 5 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from ShadenSmith ShadenSmith 5 years ago
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from tjruwase tjruwase 5 years ago
remove seq-len from transformer config
06743480
pad seq-len to be 16-aligned
0659f928
resolve the issue with softmax forward when sequence is low
cb15de6b
jeffra Merge branch 'master' into transformer/support-arbitrary-seqlen
9981c21d
jeffra
jeffra approved these changes on 2020-12-11
RezaYazdaniAminabadi
RezaYazdaniAminabadi
jeffra
jeffra Merge branch 'master' into transformer/support-arbitrary-seqlen
2a3b3d26
make the padding more efficient
3b34bcca
jeffra Merge branch 'master' into transformer/support-arbitrary-seqlen
57b01e46
jeffra bump DSE to support this PR
23c70a3b
jeffra jeffra merged fd2f970b into master 5 years ago
zmxdream
mrwyattii mrwyattii deleted the transformer/support-arbitrary-seqlen branch 2 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum