vllm-project/vllm

Pull Requests Commits

[Attention][TokenSpeed MLA] Also warm up prefill kernel from decode impl

zyongye committed 25 days ago

58c8a5ea

zyongye committed 25 days ago

c4547482

adding to cuda.txt dependency

zyongye committed 25 days ago

91ef0afc

zyongye committed 25 days ago

97cd2c41

[Attention][TokenSpeed MLA] Force prefill V tensor contiguous before kernel call

zyongye committed 25 days ago

c0015350

[Attention][TokenSpeed MLA] Surface install hint when package missing

zyongye committed 25 days ago

de6bc297

[Attention][TokenSpeed MLA] Fix trtllm LSE parity test: log2 → natural log

zyongye committed 25 days ago

00128182

[Attention][TokenSpeed MLA] Warm up BF16 prefill compile, drop seq_lens computation

zyongye committed 25 days ago

73cd7e25

[Attention][TokenSpeed MLA] Fix decode FP8 numerics: pass output_scale and assert FP8 Q

zyongye committed 25 days ago

964c6eb4

[Attention] Add TOKENSPEED_MLA backend for DeepSeek R1 prefill + decode on Blackwell

zyongye committed 25 days ago

d0e6514b

[Frontend] Supports resubmitting output items with missing fields in Responses API (#41355)

chaunceyjiang committed 25 days ago

Verified c7aa186d

[CI] Route part of B200 jobs to b200-k8s (#41453)

khluu committed 25 days ago

Verified f6537612

[ROCm][CI] Use vLLM generation defaults for DeepSeek prefetch-offload eval (#41575)

AndreasKaratzas committed 25 days ago

Verified 4a8ae26e

[CI] Automate Docker Hub release image publishing (#40415)

khluu committed 25 days ago

Verified 13338644

[Attention] Minor refactor: layer takes ownership of the MLA prefill backend (#41744)

MatthewBonanni committed 25 days ago

Verified 01b9b5af

Bump model-hosting-container-standards to >= 0.1.14 (#39755)

Dhruvilbhatt committed 25 days ago

Verified 8c57b6e7

[Spec Decode] Fix max_model_len logging in speculative config for draft model (#41571)

liulanze committed 25 days ago

Verified 79246b5e

Fix DeepGEMM ep_scatter output address overflow (#39213)

S1ro1 committed 25 days ago

Verified 48954de2

[BUGFIX] Support streamed_args_for_tool in MistralToolParser (#41730)

juliendenize committed 25 days ago

Verified c6235ed1

[New Model][ROCm] Add AMD support for DeepSeek V4 (#40871)

whx-sjtu committed 25 days ago

Verified 628c4363

[Attention] Move FA3→FA4 upgrade into get_flash_attn_version() (#40815)

gcanlin committed 25 days ago

Verified 2228fe68

Remove unnecessary runtime asserts from linear layers (#41729)

hmellor committed 25 days ago

Verified 84bd8a3c

[Bugfix] Suggest upgrading Transformers for tokenizer class errors (#38099)

Lidang-Jiang committed 25 days ago

Verified b786ec8e

[Bugfix] Fix `RuntimeError: Already borrowed` by adding thread-safe Hugging Face fast-tokenizer wrappers (#41181)

yzong-rh committed 25 days ago

Verified 20dcd984

[BugFix][MyPy]: Module has no attribute "sched_getaffinity" [attr-defined] (#41465)

hickeyma committed 25 days ago

Verified 6fca5181

[Bugfix][KVConnector] Support DCP/PCP in OffloadingConnector (#41549)

Etelis committed 25 days ago

Verified 98661fe0

Fix DeepSeek-OCR for Transformers v4 (#41460)

hmellor committed 25 days ago

Verified b0765bee

[Model] support Qianfan-OCR model (#40136)

marvinzh committed 25 days ago

Verified 0a201b60

[Feature] Add Triton kernel JIT compilation monitor for inference (#40137)

arpera committed 26 days ago

Verified 8b9ea2f8

[XPU] use xpu topk topp sample kernel (#39285)

jikunshang committed 26 days ago

Verified 2ceea429

Older