vllm-project/vllm

Pull Requests Commits

[Attention][TokenSpeed MLA] Also warm up prefill kernel from decode impl

zyongye committed 4 days ago

58c8a5ea

zyongye committed 4 days ago

c4547482

adding to cuda.txt dependency

zyongye committed 4 days ago

91ef0afc

zyongye committed 4 days ago

97cd2c41

[Attention][TokenSpeed MLA] Force prefill V tensor contiguous before kernel call

zyongye committed 4 days ago

c0015350

[Attention][TokenSpeed MLA] Surface install hint when package missing

zyongye committed 4 days ago

de6bc297

[Attention][TokenSpeed MLA] Fix trtllm LSE parity test: log2 → natural log

zyongye committed 4 days ago

00128182

[Attention][TokenSpeed MLA] Warm up BF16 prefill compile, drop seq_lens computation

zyongye committed 4 days ago

73cd7e25

[Attention][TokenSpeed MLA] Fix decode FP8 numerics: pass output_scale and assert FP8 Q

zyongye committed 4 days ago

964c6eb4

[Attention] Add TOKENSPEED_MLA backend for DeepSeek R1 prefill + decode on Blackwell

zyongye committed 4 days ago

d0e6514b

[Frontend] Supports resubmitting output items with missing fields in Responses API (#41355)

chaunceyjiang committed 4 days ago

Verified c7aa186d

[CI] Route part of B200 jobs to b200-k8s (#41453)

khluu committed 4 days ago

Verified f6537612

[ROCm][CI] Use vLLM generation defaults for DeepSeek prefetch-offload eval (#41575)

AndreasKaratzas committed 4 days ago

Verified 4a8ae26e

[CI] Automate Docker Hub release image publishing (#40415)

khluu committed 4 days ago

Verified 13338644

[Attention] Minor refactor: layer takes ownership of the MLA prefill backend (#41744)

MatthewBonanni committed 4 days ago

Verified 01b9b5af

Bump model-hosting-container-standards to >= 0.1.14 (#39755)

Dhruvilbhatt committed 4 days ago

Verified 8c57b6e7

[Spec Decode] Fix max_model_len logging in speculative config for draft model (#41571)

liulanze committed 4 days ago

Verified 79246b5e

Fix DeepGEMM ep_scatter output address overflow (#39213)

S1ro1 committed 4 days ago

Verified 48954de2

[BUGFIX] Support streamed_args_for_tool in MistralToolParser (#41730)

juliendenize committed 4 days ago

Verified c6235ed1

[New Model][ROCm] Add AMD support for DeepSeek V4 (#40871)

whx-sjtu committed 4 days ago

Verified 628c4363

[Attention] Move FA3→FA4 upgrade into get_flash_attn_version() (#40815)

gcanlin committed 4 days ago

Verified 2228fe68

Remove unnecessary runtime asserts from linear layers (#41729)

hmellor committed 4 days ago

Verified 84bd8a3c

[Bugfix] Suggest upgrading Transformers for tokenizer class errors (#38099)

Lidang-Jiang committed 5 days ago

Verified b786ec8e

[Bugfix] Fix `RuntimeError: Already borrowed` by adding thread-safe Hugging Face fast-tokenizer wrappers (#41181)

yzong-rh committed 5 days ago

Verified 20dcd984

[BugFix][MyPy]: Module has no attribute "sched_getaffinity" [attr-defined] (#41465)

hickeyma committed 5 days ago

Verified 6fca5181

[Bugfix][KVConnector] Support DCP/PCP in OffloadingConnector (#41549)

Etelis committed 5 days ago

Verified 98661fe0

Fix DeepSeek-OCR for Transformers v4 (#41460)

hmellor committed 5 days ago

Verified b0765bee

[Model] support Qianfan-OCR model (#40136)

marvinzh committed 5 days ago

Verified 0a201b60

[Feature] Add Triton kernel JIT compilation monitor for inference (#40137)

arpera committed 5 days ago

Verified 8b9ea2f8

[XPU] use xpu topk topp sample kernel (#39285)

jikunshang committed 5 days ago

Verified 2ceea429

Older