Commits vllm-project/vllm

Use cumem allocator for the KV cache by default

tlrmchlsmth committed 67 days ago

Verified 5b4f6d52

[ROCm][CI] Stabilize ModernBERT token-classification parity against Hugging Face (#44040)

AndreasKaratzas committed 68 days ago

Verified b6971198

[WIP][XPU] upgrade torch-xpu to 2.12 (#42262)

jikunshang committed 68 days ago

Verified b4c6dc64

Remove `raw_inputs` from transformers backend (#39425)

zucchini-nlp committed 68 days ago

Verified 2ee51063

[Perf] fuse qk rmsnorm rope gate for qwen3.5 (#44176)

ZJY0516 committed 68 days ago

Verified 7a89b725

[Bugfix] Fix minimax_qk_norm_fusion (#44983)

jeejeelee committed 68 days ago

Verified dc10e467

[Cohere] Cohere2 moe parser fix (#44907)

Terrencezzj committed 68 days ago

Verified ee4d7df2

[Cohere] Fix Cohere2MoE weight loading when using Transformers ≥5.10 (#44747)

Terrencezzj committed 68 days ago

Verified 3e8afdf7

[PD][Bugfix] Fix KV Cache sharing with HMA (#44629)

NickLucche committed 68 days ago

Verified 6690a0c4

[Rust Frontend] Support Kimi K2 tool call IDs (#44901)

cinnamonica02 committed 68 days ago

Verified 1c23c420

[XPU][CI] Refine docker image build and pull/create lock mechanism in Intel GPU CI (#44481)

zxd1997066 committed 68 days ago

Verified b12e42d1

[Rust Frontend] Add /tokenize and /detokenize endpoints (#44222)

TanNgocDo committed 68 days ago

Verified 69fdaffb

[ROCm][Compile] Fuse AR + RMSNorm + per-group FP8 quant (+ DSv3.2 indexer fan-out) (#42864)

maeehart committed 68 days ago

Verified 80e2c446

[KV Events] Switch event structs from array to map encoding (#42892)

sagearc committed 68 days ago

Verified 5b3807e8

[Kernel][Perf] Tune fused_moe FP8 config for Qwen3-Next-80B tp=4 on H100 (+25% at batch 96-512) (#44830)

qyYue1389 committed 68 days ago

Verified 59401ac9

[Rust Frontend] Support API key authentication (#44321)

ricky-chaoju committed 68 days ago

Verified d841386d

[CI/Docs] Remove stale disagg prefill links (#44918)

mmangkad committed 68 days ago

Verified fff9210b

[DSV4][XPU] Add MHC fused_post_pre support (#44144)

majian4work committed 68 days ago

Verified 70db1488

[ROCm][CI] Stabilize sleep-mode memory release (#43022)

AndreasKaratzas committed 68 days ago

Verified 2385e140

[Bugfix][CI] Fix `test_offloading_connector.py::test_fs_tiering_offloading` (#44903)

NickLucche committed 68 days ago

Verified dab60fc6

[CI] Reorganize entrypoints CI (#44947)

noooop committed 68 days ago

Verified 996222f4

[Bugfix][MiniCPM-o] Fix cuda/cpu device mismatch in Resampler2_5 pos_embed (#43844)

parthash0804 committed 68 days ago

Verified e6fc848d

[Bugfix][CI] Gemma3 Transformers multimodal encoder profiling and build prompt-embedding fixtures (#44952)

AndreasKaratzas committed 68 days ago

Verified f843ac1a

fix: prevent MM cache hang from stale LRU order keys (#43595)

jeffye-dev committed 68 days ago

Verified 7c2aa310

[Bugfix][Rust Frontend] Set a structured-output backend so requests do not 500 (#44729)

Sunt-ing committed 68 days ago

Verified ebf53ba3

[ROCm][MLA][Bugfix] Reserve FP8 prefill workspace before lock for Kimi-K2.5 (#42978)

xaguilar-amd committed 68 days ago

Verified baacbfce

[Bugfix] Propagate ImportError from load_audio_pyav when vllm[audio] … (#44750)

littlecircle0730 committed 68 days ago

Verified d8218b1e

[MM][Perf][CG] Support ViT full CUDA graph for glm4_1v image and video inference (#40576)

grYe99 committed 68 days ago

Verified 9f153aa7

[XPU][CI] fix test case path (#44940)

jikunshang committed 68 days ago

Verified d3de6150

[Bugfix][Model] Qwen3-Omni: move cu_seqlens to GPU before VIT attention (#44264)

liulanze committed 68 days ago

Verified 540aaf21