vllm
[Attention][Perf][Kernel] Replace torch.cat with vectorized CUDA kernel MLA query concat - DeepSeek-V3.2
#34917

Merged

[Attention][Perf][Kernel] Replace torch.cat with vectorized CUDA kernel MLA query concat - DeepSeek-V3.2 #34917

vllm-bot merged 17 commits into vllm-project:main from LopezCastroRoberto:concat_mla_q

add custom concat kernel

e9831552

LopezCastroRoberto requested a review from

pavanimajety 80 days ago

LopezCastroRoberto changed the title ~~add custom concat kernel~~ [Perf] Replace torch.cat with vectorized CUDA kernel for MLA query concat 80 days ago

LopezCastroRoberto marked this pull request as draft 80 days ago

mergify added nvidia

mergify added v1

LopezCastroRoberto changed the title ~~[Perf] Replace torch.cat with vectorized CUDA kernel for MLA query concat~~ [Perf][WIP] Replace torch.cat with vectorized CUDA kernel for MLA query concat 80 days ago

gemini-code-assist commented on 2026-02-19

add tests

0ce15f9d

add helper vec instructions

09150f91

LopezCastroRoberto changed the title ~~[Perf][WIP] Replace torch.cat with vectorized CUDA kernel for MLA query concat~~ [Perf][WIP] Replace torch.cat with vectorized CUDA kernel MLA query concat - DeepSeek-V3.2 75 days ago

LopezCastroRoberto marked this pull request as ready for review 75 days ago

LopezCastroRoberto requested a review from

mgoin 75 days ago

LopezCastroRoberto requested a review from

tlrmchlsmth 75 days ago

LopezCastroRoberto requested a review from

WoosukKwon 75 days ago

LopezCastroRoberto requested a review from

yewentao256 75 days ago

mergify added deepseek

adding benchmark script

5a8fe250

adding benchmark script

9b5c159b

mergify added performance

tune threadblock size

71d00a2c

LopezCastroRoberto changed the title ~~[Perf][WIP] Replace torch.cat with vectorized CUDA kernel MLA query concat - DeepSeek-V3.2~~ [Perf] Replace torch.cat with vectorized CUDA kernel MLA query concat - DeepSeek-V3.2 74 days ago

LopezCastroRoberto changed the title ~~[Perf] Replace torch.cat with vectorized CUDA kernel MLA query concat - DeepSeek-V3.2~~ [Attention][Perf][Kernel] Replace torch.cat with vectorized CUDA kernel MLA query concat - DeepSeek-V3.2 74 days ago

mgoin added ready

Merge branch 'main' into concat_mla_q

8d05e603

mergify added needs-rebase

LucasWilkinson approved these changes on 2026-03-03

overlap with MoE workspace

8c0e7908

Merge branch 'main' into concat_mla_q

0e922863

LopezCastroRoberto requested a review from

LucasWilkinson 67 days ago

mergify removed needs-rebase

add missing instructions to vec_utils

36baa4c3

remove return amd path

75a5e371

Merge branch 'main' into concat_mla_q

605d23dc

Merge branch 'main' into concat_mla_q

8a4124c8

Merge branch 'main' into concat_mla_q

0dba242f

LucasWilkinson enabled auto-merge (squash) 65 days ago

mergify added needs-rebase

Merge branch 'main' into concat_mla_q

6c4a0420

disabled auto-merge 62 days ago
Head branch was pushed to by a user without write access

mergify removed needs-rebase

add AMD tests to CI

6f8bb714

mergify added ci/build

LopezCastroRoberto force pushed to 6f8bb714 62 days ago

remove amd ci

43dffba1

mgoin approved these changes on 2026-03-09

vllm-bot merged 580864d8 into main 62 days ago

Reviewers

mgoin

LucasWilkinson

gemini-code-assist

pavanimajety

tlrmchlsmth

WoosukKwon

yewentao256

Assignees

No one assigned

Labels

performance ready ci/build v1 deepseek nvidia

Milestone

No milestone

vllm [Attention][Perf][Kernel] Replace torch.cat with vectorized CUDA kernel MLA query concat - DeepSeek-V3.2 #34917 Merged

[Attention][Perf][Kernel] Replace torch.cat with vectorized CUDA kernel MLA query concat - DeepSeek-V3.2 #34917

vllm
[Attention][Perf][Kernel] Replace torch.cat with vectorized CUDA kernel MLA query concat - DeepSeek-V3.2
#34917

Merged