vllm
[Kernel][Performance] Fuse float cast and renormalize to topk softmax kernel
#26717

Merged

[Kernel][Performance] Fuse float cast and renormalize to topk softmax kernel #26717

mgoin merged 10 commits into vllm-project:main from izhuhaoran:fuse-topk-softmax

perf: fuse to64 and renormalize to topk softmax kernel

f0abf853

fix lint error

a0ca4e7b

izhuhaoran requested a review from

mgoin 62 days ago

gemini-code-assist commented on 2025-10-13

chatgpt-codex-connector commented on 2025-10-13

Merge branch 'main' into fuse-topk-softmax

2d64ff17

Merge branch 'main' into fuse-topk-softmax

28a11a52

mgoin commented on 2025-10-15

Merge branch 'main' into fuse-topk-softmax

969c425b

fix: use TORCH_CHECK for int64 dtype of topk_indices

34cfff69

feat: support float16 for fused_topk_softmax

b20c66b3

refactor: use template dispatch_topk_softmax_launch for diff gating_o…

02eb929c

refactor: remove unused assert for toFloat in topk_softmax

dac7677e

izhuhaoran requested a review from

mgoin 59 days ago

mgoin approved these changes on 2025-10-16

mgoin enabled auto-merge (squash) 59 days ago

github-actions added ready

Merge branch 'main' into fuse-topk-softmax

0bd79b64

mgoin merged 75c7ad99 into main 58 days ago

Reviewers

mgoin

gemini-code-assist

Assignees

No one assigned

Labels

ready

Milestone

No milestone