onnxruntime
[CUDA] Update fused MHA to support flash attention and causal mask
#13953
Merged
Overview
Commits
20
Changes
View On GitHub

[CUDA] Update fused MHA to support flash attention and causal mask #13953

tianleiwu merged 20 commits into main from tlwu/trt_flash_attention
tianleiwu
tianleiwu FMHA supports flash attention and causal
30d9b426
tianleiwu Fix unrelated build error
868547cd
tianleiwu refactoring
23da0d40
tianleiwu tianleiwu marked this pull request as draft 3 years ago
tianleiwu Fix causal fall back condition
50622274
tianleiwu Merge branch 'main' into tlwu/trt_flash_attention
2ae808c7
tianleiwu disable Attention_Mask1D_Fp16_B2_FusedNoPadding
14405a7c
tianleiwu tianleiwu requested a review from yufenglee yufenglee 3 years ago
tianleiwu tianleiwu requested a review from gh-yewang gh-yewang 3 years ago
tianleiwu Merge branch 'main' into tlwu/trt_flash_attention
df5b4bf2
tianleiwu tianleiwu marked this pull request as ready for review 3 years ago
yufenglee
yufenglee
yufenglee commented on 2022-12-22
yufenglee
yufenglee commented on 2022-12-22
yufenglee
yufenglee dismissed these changes on 2022-12-22
tianleiwu fix static analysis warning
218f2c54
tianleiwu tianleiwu dismissed their stale review via 218f2c54 3 years ago
yufenglee
yufenglee dismissed these changes on 2022-12-23
tianleiwu Do not use fused kernel in training builds
24fbe693
tianleiwu tianleiwu dismissed their stale review via 24fbe693 3 years ago
tianleiwu
tianleiwu commented on 2022-12-23
tianleiwu fix build
fa53969f
tianleiwu Add TensorRT flash attention kernels
b2df7373
tianleiwu fix unroll list
b80bf153
tianleiwu integrate trt flash attention
6674fc62
tianleiwu fix build
eb3015da
tianleiwu Merge branch 'main' into tlwu/trt_flash_attention_v2
115bbcfd
tianleiwu remove non flash kernel with S=512
874e23cb
tianleiwu clean up code
09d0c6da
tianleiwu tianleiwu marked this pull request as draft 3 years ago
tianleiwu consolidate flash attention kernels
1c7a2ffe
tianleiwu move causal kernels to sub-directories
4d80d83d
tianleiwu support sm89
7e88f16d
tianleiwu tianleiwu marked this pull request as ready for review 3 years ago
gh-yewang
gh-yewang approved these changes on 2022-12-30
tianleiwu tianleiwu merged 6a9dc6c9 into main 3 years ago
tianleiwu tianleiwu deleted the tlwu/trt_flash_attention branch 3 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum