vllm
[FEAT][ROCm]: Support AITER MLA on V1 Engine
#17523
Merged
Overview
Commits
48
Changes
View On GitHub

[FEAT][ROCm]: Support AITER MLA on V1 Engine #17523

DarkLight1337 merged 48 commits into vllm-project:main from EmbeddedLLM:aiter-mla-v1
vllmellm
vllmellm add AITER MLA implementation in attention backend
f782c664
vllmellm remove unused arguments in aiter mla decode fwd kernel
42d5c620
vllmellm add unittest for AITER MLA backend in attention selector
565a3fda
vllmellm add unittest for MLA attention backend selector
645f4009
vllmellm code cleaning
22c87260
vllmellm update AITER version
5dc13489
vllmellm Merge remote-tracking branch 'origin/main' into aiter-mla-integration
12f80237
vllmellm add ck flash attn in prefill mla computation
da8c69f9
vllmellm further code cleaning
1ea5718e
vllmellm Merge remote-tracking branch 'origin/main' into aiter-mla-integration
681d7772
vllmellm fix mypy typing errors
9ada055a
vllmellm Merge remote-tracking branch 'origin/main' into aiter-mla-integration
1ceb3b97
vllmellm fix mypy error on Iterable typing error
20a3f074
vllmellm remove padding for v tensor in AITER MLA which improves performance
194a42a1
vllmellm upgrade aiter package version
a9a02d59
vllmellm only support AITER FA in AITER MLA backend to avoid latency caused by…
02a4fb32
vllmellm Merge remote-tracking branch 'origin/main' into aiter-mla-integration
95213e29
vllmellm add missing data types of arguments in aiter_mla_decode_fwd
6e484337
vllmellm support AITER MLA backend on V1
0265f201
vllmellm uncomment the required packages in common.txt
693c8709
vllmellm bugfix in building decode metadata for AITER MLA decode forward pass
a5a1a54e
vllmellm optimize the AITER decode metadata build
38c67c76
vllmellm Merge remote-tracking branch 'origin/main' into aiter-mla-v1
74c9cb3a
vllmellm bugfix caused by merging with main
643d07f4
vllmellm Handle v1 AITER MLA backend in rocm platform
6171e508
vllmellm update AITER MLA decode metadata build
905cec93
vllmellm update AITER commit
20e769ee
vllmellm Merge remote-tracking branch 'origin/main' into aiter-mla-v1
455bbf2b
vllmellm update proper logging info in selected backend as well as updating at…
90daf6ed
vllmellm vllmellm requested a review from tlrmchlsmth tlrmchlsmth 1 year ago
vllmellm vllmellm requested a review from WoosukKwon WoosukKwon 1 year ago
vllmellm vllmellm requested a review from robertgshaw2-redhat robertgshaw2-redhat 1 year ago
vllmellm vllmellm requested a review from njhill njhill 1 year ago
vllmellm vllmellm requested a review from ywang96 ywang96 1 year ago
vllmellm vllmellm requested a review from comaniac comaniac 1 year ago
vllmellm vllmellm requested a review from alexm-redhat alexm-redhat 1 year ago
github-actions
mergify mergify added ci/build
mergify mergify added v1
vllmellm vllmellm marked this pull request as draft 1 year ago
vllmellm fix wrong sync merge to main
f68e9265
vllmellm fix pre-commit
821f475c
vllmellm Merge remote-tracking branch 'origin/main' into aiter-mla-v1
1a6ba998
vllmellm add the missing line in common.py
7cc28d28
vllmellm vllmellm marked this pull request as ready for review 1 year ago
vllmellm vllmellm marked this pull request as draft 1 year ago
vllmellm vllmellm marked this pull request as ready for review 1 year ago
hongxiayang hongxiayang added rocm
vllmellm fix wrong logger info message
29fc0600
houseroad
houseroad approved these changes on 2025-05-05
houseroad houseroad added ready
houseroad houseroad enabled auto-merge (squash) 1 year ago
vllmellm clean code and fix AITER block scaled kernel fake impl in v1 engine
7f1ed779
disabled auto-merge 1 year ago
Head branch was pushed to by a user without write access
vllmellm use env variable to adjsut timeout for model execution
11a89852
vllmellm remove unnecessary backend check
825f387c
vllmellm make model execution timeout in envs variable rocm specific variable …
cbeb0df5
houseroad
houseroad commented on 2025-05-05
houseroad
hongxiayang
hongxiayang approved these changes on 2025-05-02
hongxiayang
hongxiayang commented on 2025-05-05
hongxiayang
vllmellm fix unit-test
2218bbcf
vllmellm Merge remote-tracking branch 'origin/main' into aiter-mla-v1
cb985044
vllmellm bugfix to update AITER MLA V1 decode forward after sync with main
44d813f2
vllmellm Update vllm/platforms/rocm.py
423c0bef
vllmellm address PR comments
58d79bd5
SageMoore
SageMoore requested changes on 2025-05-06
vllmellm update assertion message
95644ea8
mergify
mergify mergify added needs-rebase
vllmellm remove env variable for model execution timeout
f41d616c
mergify mergify removed needs-rebase
vllmellm Merge remote-tracking branch 'origin/main' into aiter-mla-v1
56d22546
vllmellm remove unnecessary warning
3ee787ec
vllmellm keep model execution timeout as original value in main branch
f6884187
SageMoore
SageMoore approved these changes on 2025-05-08
DarkLight1337 DarkLight1337 merged 3c9396a6 into main 1 year ago
chaunceyjiang
tjtanaa

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
rocm ready ci/build v1
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum