vllm
[Kernel] Fuse FP8 output quantization into merge_attn_states
#36518

Merged

[Kernel] Fuse FP8 output quantization into merge_attn_states #36518

ProExpertProg merged 15 commits into vllm-project:main from carlyou:fuse-fp8-quant-merge-attn-states

carlyou requested a review from

mgoin 147 days ago

carlyou requested a review from

tlrmchlsmth 147 days ago

carlyou requested a review from

WoosukKwon 147 days ago

carlyou requested a review from

yewentao256 147 days ago

carlyou requested a review from

LucasWilkinson 147 days ago

carlyou requested a review from

MatthewBonanni 147 days ago

mergify added v1

gemini-code-assist commented on 2026-03-09

chatgpt-codex-connector commented on 2026-03-09

carlyou commented on 2026-03-09

carlyou requested a review from

russellb 145 days ago

carlyou requested a review from

tjtanaa 145 days ago

carlyou requested a review from

gshtras 145 days ago

carlyou requested a review from

sighingnow 145 days ago

carlyou requested a review from

hmellor 145 days ago

carlyou requested a review from

DarkLight1337 145 days ago

carlyou requested a review from

ywang96 145 days ago

carlyou requested a review from

njhill 145 days ago

carlyou requested a review from

tdoublep 145 days ago

carlyou requested a review from

pavanimajety 145 days ago

carlyou requested a review from

aarnphm 145 days ago

carlyou requested a review from

chaunceyjiang 145 days ago

carlyou requested a review from

youkaichao 145 days ago

carlyou requested a review from

robertgshaw2-redhat 145 days ago

carlyou requested a review from

houseroad 145 days ago

carlyou requested a review from

ProExpertProg 145 days ago

carlyou requested a review from

zou3519 145 days ago

carlyou requested a review from

BoyuanFeng 145 days ago

mergify added documentation

mergify added ci/build

mergify added llama

mergify added performance

mergify added qwen

mergify added nvidia

mergify added rocm

carlyou force pushed 145 days ago

ProExpertProg commented on 2026-03-12

carlyou force pushed to 4930ca23 142 days ago

ProExpertProg commented on 2026-03-14

carlyou commented on 2026-03-16

carlyou force pushed from 21c1e25e 125 days ago

ProExpertProg added ready

ProExpertProg approved these changes on 2026-04-01

carlyou force pushed 124 days ago

ProExpertProg commented on 2026-04-02

carlyou force pushed 123 days ago

[Kernel] Fuse FP8 output quantization into merge_attn_states

d966c76a

Address PR review: dtype guards, zero-scale check, dedup merge loop

3eb2d5ac

chore: update triton changes

15ca0f46

Address PR review: naming, scale inversion, comment, tests

b307e8b8

Add benchmark, parametrize FP8 test input dtypes, fix docstring

68beaee4

fix benchmark comparison for triton

6816882c

fix: deepseek head_size in benchmark

4bf09eb4

Address PR review comments

2c7d541e

fix: compute scale inverse inside triton kernel

feaceee1

chore

bdf9e46f

fix: issue from rebase

75f5c692

test: update per comment

6c2ac531

chore: touchup test case

2d422e7c

test: update precision in test

2cb6d4f7

test: update quant compare

57a8cba6

carlyou force pushed to 57a8cba6 123 days ago

ProExpertProg enabled auto-merge (squash) 123 days ago

ProExpertProg merged 3bc2734d into main 123 days ago

carlyou deleted the fuse-fp8-quant-merge-attn-states branch 122 days ago

Reviewers

ProExpertProg

chatgpt-codex-connector

gemini-code-assist

mgoin

tlrmchlsmth

WoosukKwon

yewentao256

LucasWilkinson

MatthewBonanni

russellb

tjtanaa

gshtras

sighingnow

hmellor

DarkLight1337

ywang96

njhill

tdoublep

pavanimajety

aarnphm

chaunceyjiang

youkaichao

robertgshaw2-redhat

houseroad

zou3519

BoyuanFeng

Assignees

No one assigned

Labels

documentation performance rocm ready ci/build v1 llama qwen nvidia

Milestone

No milestone

vllm [Kernel] Fuse FP8 output quantization into merge_attn_states #36518 Merged

[Kernel] Fuse FP8 output quantization into merge_attn_states #36518

vllm
[Kernel] Fuse FP8 output quantization into merge_attn_states
#36518

Merged