PR #18007 Split KV on MHA and Attention ops

Split KV on MHA and Attention ops #18007

tianleiwu merged 6 commits into main from aciddelgado/mha_splitkv

split kv on mha and att ops

e283e013

aciddelgado requested a review from

tianleiwu 2 years ago

aciddelgado requested a review from

yufenglee 2 years ago

tianleiwu commented on 2023-10-18

update num_splits heuristic

72326d8e

aciddelgado added release:1.16.2

tianleiwu commented on 2023-10-24

Merge branch 'main' into aciddelgado/mha_splitkv

868fb4b0

helper function for split buffers

9401ac9a

faxu added triage:approved

faxu added sdxl_llama

Merge branch 'main' into aciddelgado/mha_splitkv

7af984c7

lint

bd4d74c8

tianleiwu approved these changes on 2023-11-01

tianleiwu merged 819b5a3e into main 2 years ago

tianleiwu deleted the aciddelgado/mha_splitkv branch 2 years ago

tianleiwu removed triage:approved

tianleiwu removed release:1.16.2

tianleiwu removed sdxl_llama

Reviewers

tianleiwu

yufenglee

Assignees

No one assigned

Labels

None yet

Milestone

No milestone