vllm-project/vllm

Pull Requests Commits

[Spec Decode] Add Gemma4 MTP speculative decoding with centroids masking

lucianommartins committed 108 days ago

9b4e8393

[New Model][ROCm] Add AMD support for DeepSeek V4 (#40871)

whx-sjtu committed 108 days ago

Verified 628c4363

[Attention] Move FA3→FA4 upgrade into get_flash_attn_version() (#40815)

gcanlin committed 108 days ago

Verified 2228fe68

Remove unnecessary runtime asserts from linear layers (#41729)

hmellor committed 108 days ago

Verified 84bd8a3c

[Bugfix] Suggest upgrading Transformers for tokenizer class errors (#38099)

Lidang-Jiang committed 108 days ago

Verified b786ec8e

[Bugfix] Fix `RuntimeError: Already borrowed` by adding thread-safe Hugging Face fast-tokenizer wrappers (#41181)

yzong-rh committed 108 days ago

Verified 20dcd984

[BugFix][MyPy]: Module has no attribute "sched_getaffinity" [attr-defined] (#41465)

hickeyma committed 108 days ago

Verified 6fca5181

[Bugfix][KVConnector] Support DCP/PCP in OffloadingConnector (#41549)

Etelis committed 108 days ago

Verified 98661fe0

Fix DeepSeek-OCR for Transformers v4 (#41460)

hmellor committed 108 days ago

Verified b0765bee

[Model] support Qianfan-OCR model (#40136)

marvinzh committed 108 days ago

Verified 0a201b60

[Feature] Add Triton kernel JIT compilation monitor for inference (#40137)

arpera committed 108 days ago

Verified 8b9ea2f8

[XPU] use xpu topk topp sample kernel (#39285)

jikunshang committed 108 days ago

Verified 2ceea429

[P/D][Mooncake] Add KVConnectorStats for transfer observability (#40414)

zhewenl committed 108 days ago

Verified bee12616

[Model] Use AutoWeightsLoader for Plamo2 (#41699)

Crystora committed 108 days ago

Verified 27cc676b

[Benchmark] Add --trust-remote-code flag to multi-turn benchmark (#41661)

Dao007forever committed 108 days ago

Verified 4845aee6

[Model] Use AutoWeightsLoader for CohereMoe (#41690)

Crystora committed 109 days ago

Verified 0c620d2e

[Model] Fix Gemma4 MoE activation mismatch (#41574)

lucianommartins committed 109 days ago

Verified 6bb924bb

[BugFix] Preserve max_seq_len in ubatch metadata during CUDA graph capture (#40961)

czhu-cohere committed 109 days ago

Verified eaec7be4

[Ray] Enable RayExecutorV2 by default (#41421)

jeffreywang88 committed 109 days ago

Verified f04fd167

[Hardware][Power]Add Power VSX Attention Backend and fix l2 Cache Crash (#40451)

Akashcodes732 committed 109 days ago

Verified 420b0a5c

[ROCm][Quantization][2/N] Refactor quark_moe w4a8 w/ oracle (#39136)

BowenBao committed 109 days ago

Verified 1e950041

[Perf][2/n] Eliminate GPU<->CPU syncs in pooling code (#41433)

njhill committed 109 days ago

Verified 416f9cde

[XPU] enable is_act_and_mul for xpu (#37481)

xuechendi committed 109 days ago

Verified 685bf811

[Model Runner V2] Rebuild attn metadata between draft decode steps (#41162)

TheEpicDolphin committed 109 days ago

Verified e1e4646b

[Feature] TurboQuant: support hybrid models and uniform quantization (#39931)

JartX committed 109 days ago

Verified 4f2af1a7

[Bug] Fix status update address for non-MOE model within external dp mode (#40839)

yewentao256 committed 109 days ago

Verified 577b9623

[ROCm][CI] Fix MLA prefill scale for DeepSeek GSM8K (#41569)

AndreasKaratzas committed 109 days ago

Verified 1cb08387

[Docs] Add non-causal support to attention backend docs (#41643)

MatthewBonanni committed 109 days ago

Verified be5983b8

[NVFP4][fix] Fix `layer.weight` -> `w13` typo in NVFP4 MOE emulation kernel preparation (#41630)

fxmarty-amd committed 109 days ago

Verified 9c07342f

feat: update xgrammar==0.2.0 to use structural tags for strict tool calling + reasoning for more models (#40894)

Seven-Streams committed 109 days ago

Verified 844df542

Older