onnxruntime
Support group query attention in Attention(23) CUDA
#27082

Merged

Support group query attention in Attention(23) CUDA #27082

titaiwangms merged 28 commits into main from copilot/support-group-query-attention

Initial plan

e538f4be

copilot-swe-agent assigned

copilot-swe-agent 242 days ago

copilot-swe-agent assigned

titaiwangms 242 days ago

Add GQA support to Attention(23) CUDA operator

f2007a2e

Add debug tracking fields and num_splits parameter

53c333fd

Fix code review issues: use v_head_size and parameters.softcap

042ff320

Set softcap to 0.0f explicitly with comment

0e7a632d

copilot-swe-agent changed the title ~~[WIP] Implement support for group query attention in Attention(23)~~ Support group query attention in Attention(23) CUDA 242 days ago

copilot-swe-agent requested a review from

titaiwangms 242 days ago

Enable CUDA tests for GQA attention tests

2e108741

Remove GQA test filters from disabled tests list

b86acbd1

Add float template instantiation for GQA QkvToContext

213a82d2

Revert float support for GQA and add type validation

f79c509a

titaiwangms added ep:CUDA

change gqa tests to fp16

e52efb25

examine gqa parameters and move down MHA parameters

98c5dcfe

Merge branch 'main' into copilot/support-group-query-attention

0f800f5d

support gqa bool masking

4978e96f

add flash/memory draft

4c644e2e

Merge branch 'main' into copilot/support-group-query-attention

f04b38e4

finish gqa default

16d54531

titaiwangms commented on 2026-01-28

Apply suggestion from @titaiwangms

54d77ae3

introduce python attention tests for gqa

87a56484

github-advanced-security commented on 2026-01-28

github-actions commented on 2026-01-28

github-advanced-security commented on 2026-01-28

lint

59810416

support attn_mask

6d7e50ad

Merge branch 'main' into copilot/support-group-query-attention

d1cb0635

titaiwangms marked this pull request as ready for review 233 days ago

titaiwangms commented on 2026-01-30

clean up and use ORT_MAKE_STATUS

e2a40324

Merge branch 'main' into copilot/support-group-query-attention

dcb937ab

titaiwangms commented on 2026-01-31

fix cpu bugs on fp16

25094644

titaiwangms requested a review from

tianleiwu 228 days ago

Merge branch 'main' into copilot/support-group-query-attention

3b17c71e

Merge branch 'main' into copilot/support-group-query-attention

cd992ee4

Merge branch 'main' into copilot/support-group-query-attention

20c02d37

update tests

1634cdd3

tianleiwu approved these changes on 2026-02-10

xadupre commented on 2026-02-10

copilot-swe-agent requested a review from

xadupre 221 days ago

titaiwangms approved these changes on 2026-02-10

titaiwangms closed this 221 days ago

titaiwangms reopened this 221 days ago

titaiwangms enabled auto-merge (squash) 221 days ago

disabled auto-merge 221 days ago
Manually disabled by user

titaiwangms enabled auto-merge (squash) 221 days ago

titaiwangms merged 5a9877a0 into main 220 days ago

titaiwangms deleted the copilot/support-group-query-attention branch 220 days ago

Reviewers

titaiwangms

tianleiwu

xadupre

github-actions

github-advanced-security

Assignees

titaiwangms

Copilot

Labels

ep:CUDA

Milestone

No milestone

onnxruntime Support group query attention in Attention(23) CUDA #27082 Merged

Support group query attention in Attention(23) CUDA #27082

onnxruntime
Support group query attention in Attention(23) CUDA
#27082

Merged