vllm
[W8A8 Block Linear Refactor][2/N] Remove W8A8Fp8BlockLinearOp and adopt Fp8 block linear kernel selections.
#33892
Merged
Overview
Commits
91
Changes
View On GitHub

[W8A8 Block Linear Refactor][2/N] Remove W8A8Fp8BlockLinearOp and adopt Fp8 block linear kernel selections. #33892

tjtanaa merged 91 commits into vllm-project:main from EmbeddedLLM:3n-block-scaled-rfc-pr
maralbahari
maralbahari create initial block scaled mm kernels and a common base
8a542b76
maralbahari remove W8A8Fp8BlockLinearOp and adop mm kernel selection
0ebcf788
mergify mergify added performance
mergify mergify added nvidia
gemini-code-assist
gemini-code-assist commented on 2026-02-05
maralbahari remove W8A8Fp8BlockLinearOp from unit tests
b76074c0
maralbahari Update vllm/model_executor/layers/quantization/kernels/base.py
3c7049ed
maralbahari Update vllm/model_executor/layers/quantization/kernels/base.py
08a893dc
maralbahari Update vllm/model_executor/layers/quantization/kernels/scaled_mm/aite…
98471098
maralbahari Update vllm/model_executor/layers/quantization/kernels/scaled_mm/cuda.py
5d589357
maralbahari Update vllm/model_executor/layers/quantization/kernels/scaled_mm/Bloc…
9887678e
maralbahari fix pre-commit issues and typings
4b536756
maralbahari imporve typing
acac7c14
maralbahari Merge remote-tracking branch 'origin/2n-block-scaled-rfc-pr' into 3n-…
61bfb5b1
mergify
mergify mergify added needs-rebase
maralbahari add missing kwargs for aiter fp8 block scaled mm func and return stat…
3363c888
maralbahari Merge remote-tracking branch 'origin/2n-block-scaled-rfc-pr' into 3n-…
79951e21
maralbahari fix f-string
6465faaf
maralbahari Merge remote-tracking branch 'origin/2n-block-scaled-rfc-pr' into 3n-…
5b3c2e18
maralbahari Merge remote-tracking branch 'origin/main' into 3n-block-scaled-rfc-pr
8dd23bd8
mergify mergify removed needs-rebase
maralbahari improve documenetation and fix typings in init_fp8_linear_kernel
320ced0c
maralbahari Merge remote-tracking branch 'origin/2n-block-scaled-rfc-pr' into 3n-…
d0cd8a23
maralbahari maralbahari marked this pull request as ready for review 160 days ago
maralbahari maralbahari requested a review from tjtanaa tjtanaa 160 days ago
maralbahari maralbahari requested a review from mgoin mgoin 160 days ago
maralbahari maralbahari requested a review from robertgshaw2-redhat robertgshaw2-redhat 160 days ago
maralbahari maralbahari requested a review from yewentao256 yewentao256 160 days ago
maralbahari maralbahari requested a review from pavanimajety pavanimajety 160 days ago
maralbahari maralbahari requested a review from tlrmchlsmth tlrmchlsmth 160 days ago
dosubot
mergify
mergify mergify added needs-rebase
maralbahari Merge remote-tracking branch 'origin/main' into 2n-block-scaled-rfc-pr
f555f757
maralbahari Merge remote-tracking branch 'origin/2n-block-scaled-rfc-pr' into 3n-…
614cef56
mergify mergify removed needs-rebase
mergify
mergify mergify added needs-rebase
maralbahari Merge remote-tracking branch 'origin/main' into 2n-block-scaled-rfc-pr
c43b6cd1
maralbahari Merge remote-tracking branch 'origin/2n-block-scaled-rfc-pr' into 3n-…
ce88d6ef
mergify mergify removed needs-rebase
maralbahari fix import error
08d6a543
mergify
maralbahari fix imports
4bc9347f
tjtanaa tjtanaa added ready
tjtanaa tjtanaa added rocm
maralbahari fix import
d001db80
maralbahari maralbahari requested a review from WoosukKwon WoosukKwon 158 days ago
maralbahari use the same variable name for inpt quantization to follow scaled_mm
de82fd1a
tjtanaa tjtanaa added ready-run-all-tests
tjtanaa
tjtanaa commented on 2026-02-25
tjtanaa
tjtanaa commented on 2026-02-25
tjtanaa
tjtanaa commented on 2026-02-25
tjtanaa
tjtanaa commented on 2026-02-25
tjtanaa
tjtanaa commented on 2026-02-25
tjtanaa
tjtanaa commented on 2026-02-25
tjtanaa
tjtanaa commented on 2026-02-25
tjtanaa
tjtanaa commented on 2026-02-25
tjtanaa
tjtanaa commented on 2026-02-25
tjtanaa
tjtanaa commented on 2026-02-25
tjtanaa
tjtanaa commented on 2026-02-25
maralbahari address PR comments
7a26e606
maralbahari Merge remote-tracking branch 'origin/2n-block-scaled-rfc-pr' into 3n-…
15c3d44f
maralbahari bugfixes
e5bbb6c0
maralbahari address PR comment
cb469794
maralbahari bugfix compressed tensors
f27d31a5
maralbahari fix unit tests
d805795e
maralbahari maralbahari changed the title [W8A8 Block Linear Refactor][3/N] Remove W8A8Fp8BlockLinearOp and adopt Fp8 block linear kernel selections. [W8A8 Block Linear Refactor][2/N] Remove W8A8Fp8BlockLinearOp and adopt Fp8 block linear kernel selections. 157 days ago
tjtanaa
tjtanaa commented on 2026-02-26
maralbahari add group_size check for cutlass and deep_gemm kernels and update fus…
bfcd5227
mergify
maralbahari fix wrong check on block fp8 cutlass can_implement
ca8b19d7
maralbahari fix potential bugs in deepgemm
00a75227
maralbahari bugfix reading correct weight_scale in block scaled mm linear
e97a4795
tjtanaa
tjtanaa commented on 2026-03-04
tjtanaa
tjtanaa commented on 2026-03-04
mergify
mergify mergify added needs-rebase
maralbahari Merge remote-tracking branch 'origin/main' into 3n-block-scaled-rfc-pr
0236228d
mergify mergify removed needs-rebase
mergify
maralbahari fix pre-commit issue
6ce17a92
maralbahari Merge remote-tracking branch 'origin/main' into 3n-block-scaled-rfc-pr
26450416
tjtanaa Merge branch 'main' into 3n-block-scaled-rfc-pr
a3d7831f
tjtanaa tjtanaa requested a review from LucasWilkinson LucasWilkinson 137 days ago
tjtanaa Merge branch 'main' into 3n-block-scaled-rfc-pr
ed5a54c9
tjtanaa
LucasWilkinson
LucasWilkinson commented on 2026-03-18
mergify
mergify mergify added needs-rebase
maralbahari initialize kernels in create_weights
c28dac9b
maralbahari Merge remote-tracking branch 'origin/main' into 3n-block-scaled-rfc-pr
2478541c
mergify mergify removed needs-rebase
maralbahari fix fusion unit tests
dba5697e
mergify
maralbahari fix fusion unit test and online fp8 quant
f0ca1e9f
mergify
maralbahari fix pre-commit error
f5951125
maralbahari fix input_dtype
55096efc
maralbahari fix unittest
64df3016
mergify
mergify mergify added needs-rebase
maralbahari Merge remote-tracking branch 'origin/main' into 3n-block-scaled-rfc-pr
a08f6230
mergify mergify removed needs-rebase
maralbahari fix unit tests
1d5c1b70
maralbahari Merge remote-tracking branch 'origin/main' into 3n-block-scaled-rfc-pr
98f215bc
maralbahari
maralbahari commented on 2026-03-24
maralbahari fix unit test for test_modelopt
1b65c2e4
maralbahari remove unused function.
f093d82c
LucasWilkinson
LucasWilkinson approved these changes on 2026-03-24
maralbahari fix Quantization unit test
cbb05991
maralbahari attemp to fix marlin fp8 quant fp8
05b7cc95
tjtanaa Merge branch 'main' into 3n-block-scaled-rfc-pr
929d05d2
tjtanaa tjtanaa enabled auto-merge (squash) 130 days ago
tjtanaa Merge branch 'main' into 3n-block-scaled-rfc-pr
8593412c
maralbahari Merge remote-tracking branch 'origin/main' into 3n-block-scaled-rfc-pr
5c73e37c
maralbahari Merge branch '3n-block-scaled-rfc-pr' of https://github.com/EmbeddedL…
a07f484e
mergify
mergify mergify added needs-rebase
maralbahari Merge remote-tracking branch 'origin/main' into 3n-block-scaled-rfc-pr
f3a1cd2d
disabled auto-merge 129 days ago
Head branch was pushed to by a user without write access
mergify mergify removed needs-rebase
mergify
mergify mergify added needs-rebase
maralbahari fix deepgemm ep2 accuracy issue
c86b1722
maralbahari Merge remote-tracking branch 'origin/main' into 3n-block-scaled-rfc-pr
cf0618c0
mergify mergify removed needs-rebase
tjtanaa Merge branch 'main' into 3n-block-scaled-rfc-pr
7930f5ac
maralbahari Merge branch 'main' into 3n-block-scaled-rfc-pr
f01ba9e4
maralbahari avoid calling is_flashinfer_fp8_blockscale_gemm_supported as class va…
ad95ceb2
maralbahari Merge branch '3n-block-scaled-rfc-pr' of https://github.com/EmbeddedL…
f5348e82
maralbahari fix torch compile issue with torch.cond
f012fae6
maralbahari fix torch.cond torch.compile errors
99859191
vllmellm Merge branch 'main' into 3n-block-scaled-rfc-pr
9520a254
maralbahari bugfix wrong input quantization
ab00fb7d
maralbahari Merge remote-tracking branch 'origin/main' into 3n-block-scaled-rfc-pr
594fc5b6
mergify
mergify mergify added needs-rebase
maralbahari fix torch.cond fx-graph break
ab2d3fc7
maralbahari Merge remote-tracking branch 'origin/main' into 3n-block-scaled-rfc-pr
a3041f77
mergify mergify removed needs-rebase
maralbahari fix mxfp8 test fail
42f33345
maralbahari clean code
697d7472
mergify
maralbahari Merge remote-tracking branch 'origin/main' into 3n-block-scaled-rfc-pr
0498d028
maralbahari fix new attention mla fusion unit test
af9ec825
maralbahari fix wrong skip condition
f79b1fd1
maralbahari fix mxfp8 unit test
5e76e753
maralbahari maybe fix cutlass block scaled gemm
24a4f257
maralbahari clean code
7e254a29
maralbahari Merge remote-tracking branch 'origin/main' into 3n-block-scaled-rfc-pr
46ffd257
maralbahari fix batch-invariant issue
186fe8f2
maralbahari Merge remote-tracking branch 'origin/main' into 3n-block-scaled-rfc-pr
80565223
mergify
maralbahari fix online fp8
9cb0ebf4
mergify
maralbahari fix pre-commit
bb9920fa
maralbahari fix mxfp8 linearmethod
884b952c
maralbahari fix pytorch compile test
8e6b3aba
maralbahari maralbahari requested a review from zou3519 zou3519 118 days ago
maralbahari maralbahari requested a review from youkaichao youkaichao 118 days ago
maralbahari maralbahari requested a review from ProExpertProg ProExpertProg 118 days ago
maralbahari maralbahari requested a review from BoyuanFeng BoyuanFeng 118 days ago
maralbahari maralbahari requested a review from vadiklyutiy vadiklyutiy 118 days ago
tjtanaa Merge branch 'main' into 3n-block-scaled-rfc-pr
1a88320c
tjtanaa Merge branch 'main' into 3n-block-scaled-rfc-pr
5b572ac9
tjtanaa Merge branch 'main' into 3n-block-scaled-rfc-pr
92ba6775
tjtanaa
tjtanaa approved these changes on 2026-04-09
tjtanaa tjtanaa merged 2e9034c9 into main 115 days ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
performance rocm ready nvidia ready-run-all-tests
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum