onnxruntime
Decoder Attention CUDA Op
#9792
Merged
Overview
Commits
28
Changes
View On GitHub

Decoder Attention CUDA Op #9792

gh-yewang merged 28 commits into master from wangye/decoder_attn_pr
gh-yewang
gh-yewang add kernel interface
d0d6cda3
gh-yewang register kernel
28ec5dc0
gh-yewang add self/cross qkv projection without cache
b9048143
gh-yewang add LaunchTransQkv2 for (S,B,X,N,H) -> (X,B,N,S,H)
becd565d
gh-yewang refactor ConcatPastToPresent
88fe5e76
gh-yewang DecoderQkvToContext interface
11f10a3a
gh-yewang q,k,v buffer and cache as output
d0cec212
gh-yewang qk, pv and transctx
96100882
gh-yewang fix compiler error on linux machine
738ac483
gh-yewang key_padding_mask
0c14142a
gh-yewang add test_parity file. However not runnable
452ae522
gh-yewang add partial unittest
8aeff2a0
gh-yewang made partial attributes to inputs
eb5e3eae
gh-yewang --gen_doc
f5f83a97
gh-yewang change kernel interface, add more tests
a0c12c8d
gh-yewang morre parity tests
7fe75ae2
gh-yewang fix test
54d04018
gh-yewang fix typo
54f59846
gh-yewang transpose optimizer has bug. remove it temporarily
960a3b23
gh-yewang add input shape checks
d7f1d57d
gh-yewang add type/shape inference
a332396d
gh-yewang Merge branch 'master' into wangye/decoder_attn_pr
6dcb0006
gh-yewang fix cache shape check
a95dd438
gh-yewang Merge branch 'wangye/decoder_attn_pr' of github.com:microsoft/onnxrun…
bd3eb455
gh-yewang gh-yewang marked this pull request as ready for review 4 years ago
gh-yewang gh-yewang requested a review from hanbitmyths hanbitmyths 4 years ago
hanbitmyths
hanbitmyths commented on 2021-11-18
gh-yewang
gh-yewang fix rocm build failure
a207efb1
gh-yewang gh-yewang requested a review from hanbitmyths hanbitmyths 4 years ago
gh-yewang fix rocm build error
b5d205f1
hanbitmyths
hanbitmyths commented on 2021-11-19
hanbitmyths
hanbitmyths commented on 2021-11-19
hanbitmyths
hanbitmyths commented on 2021-11-19
hanbitmyths
hanbitmyths commented on 2021-11-19
hanbitmyths
hanbitmyths commented on 2021-11-19
hanbitmyths
hanbitmyths commented on 2021-11-19
hanbitmyths
hanbitmyths commented on 2021-11-19
hanbitmyths
hanbitmyths commented on 2021-11-19
gh-yewang review comments
91fb9df4
gh-yewang gh-yewang requested a review from hanbitmyths hanbitmyths 4 years ago
hanbitmyths
hanbitmyths commented on 2021-11-19
hanbitmyths
hanbitmyths commented on 2021-11-19
gh-yewang review comments
563968e9
gh-yewang gh-yewang requested a review from hanbitmyths hanbitmyths 4 years ago
hanbitmyths
hanbitmyths approved these changes on 2021-11-20
gh-yewang gh-yewang merged 6856619b into master 4 years ago
gh-yewang gh-yewang deleted the wangye/decoder_attn_pr branch 4 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum