PR #23808 [webgpu] Flash attention for generation

[webgpu] Flash attention for generation #23808

guschmue merged 18 commits into main from attention_generate_fa

github-actions commented on 2025-02-25

guschmue added ep:WebGPU

[webgpu] Add flash decoding

f0424fd1

qjia7 force pushed from 6f6d6d15 to f0424fd1 1 year ago

fix CI errors

48affd3b

qjia7 changed the title ~~[WIP] Flash attention for generation~~ [webgpu] Flash attention for generation 1 year ago

limit it to static kv cache

96aaa897

qjia7 requested a review from

sushraja-msft 1 year ago

qjia7 requested a review from

guschmue 1 year ago

qjia7 marked this pull request as ready for review 1 year ago

Merge branch 'main' into attention_generate_fa_good

a97ad569

remove the limitations

40aa7ada

Merge branch 'main' into attention_generate_fa_good

99df2e9d

Use 1D dispatch group size

e9c18db9

qjia7 marked this pull request as draft 1 year ago

add annotations

c96e925d

qjia7 marked this pull request as ready for review 1 year ago

Use simialr var name with matmul

0fb5c2f0

Merge branch 'main' into attention_generate_fa_good

0d3a7381

update cache hints

7cbed5fa

sushraja-msft requested changes on 2025-03-26

address comments

2526992b

qjia7 commented on 2025-03-27

qjia7 requested a review from

sushraja-msft 1 year ago

sushraja-msft commented on 2025-03-31

address comments

922ca1b9

Merge branch 'main' into attention_generate_fa_good

51595805

Rename XXXSplitK to XXXSplitVxScore

2f4a1f79

Modify the comments

acbf5442

sushraja-msft approved these changes on 2025-04-02

sushraja-msft dismissed these changes on 2025-04-04

guschmue dismissed these changes on 2025-04-04

Merge branch 'main' into attention_generate_fa_good

639a2abd

address comments

191cf414

qjia7 dismissed their stale review via 191cf414 1 year ago

qjia7 commented on 2025-04-07

qjia7 requested a review from

sushraja-msft 1 year ago

qjia7 requested a review from

guschmue 1 year ago

sushanthr approved these changes on 2025-04-08

guschmue approved these changes on 2025-04-08

guschmue removed review request from

sushraja-msft 1 year ago

guschmue merged 18f91e55 into main 1 year ago

guschmue deleted the attention_generate_fa branch 1 year ago

Reviewers

guschmue

sushanthr

sushraja-msft

github-actions

Assignees

No one assigned

Labels

ep:WebGPU

Milestone

No milestone

onnxruntime [webgpu] Flash attention for generation #23808 Merged

[webgpu] Flash attention for generation #23808

onnxruntime
[webgpu] Flash attention for generation
#23808

Merged