onnxruntime
Support flash attention on 2d attention mask for gpt2 left padding.
#14215

Merged

Support flash attention on 2d attention mask for gpt2 left padding. #14215

zhanghuanrong merged 6 commits into main from zhalei/enable2dmaskForFusedMultiHeadAttention

Support flash attention on 2d attention mask for gpt2 left padding.

f3f554de

tianleiwu commented on 2023-01-10

Support both left padding and right padding for attention mask

f5f3df14

use AttentionMaskType::MASK_2D_KEY_PADDING

39c4e602

tianleiwu commented on 2023-01-10

fix typo

8e4c5260

save reduce. pass build.

4c64c910

tianleiwu commented on 2023-01-12

update the support after offline testing.

4432b15c

tianleiwu approved these changes on 2023-01-17

yufenglee added release:1.14

zhanghuanrong merged a8df6c35 into main 2 years ago

zhanghuanrong deleted the zhalei/enable2dmaskForFusedMultiHeadAttention branch 2 years ago

faxu removed release:1.14

Reviewers

tianleiwu

Assignees

No one assigned

Labels

None yet

Milestone

No milestone