onnxruntime
Implement reuse past and present tensor in Attention Ops.
#13791
Merged
Merged
Implement reuse past and present tensor in Attention Ops. #13791
zhanghuanrong
merged
16 commits into
main
from
zhalei/attention_reuse_past_present
Implement reuse kv_cache for past and present tensor in Attention Ops.
38566baf
Python file format
4e593997
tianleiwu
commented
on 2022-12-01
tianleiwu
commented
on 2022-12-01
tianleiwu
commented
on 2022-12-01
Fix vocab_size not set issue which cause random crash.
f6ec1d78
gh-yewang
commented
on 2022-12-05
yufenglee
commented
on 2022-12-05
yufenglee
commented
on 2022-12-05
yufenglee
commented
on 2022-12-06
Merge branch 'main' into zhalei/attention_reuse_past_present
2cb38017
according pr feedbacks.
9659987c
correct some on the LaunchAddBiasTransAppendKvToPresent
3a52f9fb
yufenglee
commented
on 2022-12-15
yufenglee
commented
on 2022-12-15
support past present share buffer for attention when bias is nullptr.
df2dd1b1
yufenglee
commented
on 2022-12-15
Merge branch 'main' into zhalei/attention_reuse_past_present
eb4b73c5
yufenglee
dismissed these changes
on 2022-12-15
Some condition check
aa9cca0e
yufenglee
dismissed these changes
on 2022-12-15
yufenglee
commented
on 2022-12-15
optimized matrix number on qkv
ee8b9d71
Fix CI build issues
05ffbf90
another python lint error
89ee6753
yufenglee
dismissed these changes
on 2022-12-16
Make past_present_share_buffer change do put into result model.
a7878f45
fix prefast warnings.
624afa02
update op docs.
acbd0c99
yufenglee
dismissed these changes
on 2022-12-17
Using CI pipelines doc gen, local one seems not good.
eb2dc908
hariharans29
approved these changes
on 2022-12-17
tianleiwu
commented
on 2022-12-21
tianleiwu
commented
on 2022-12-21
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub