vllm-project/vllm

Pull Requests Commits

[Core] Enable extensible KV cache for all attention backends and Mamba

zhuohan123 committed 10 hours ago

0116e1ce

zhuohan123 committed 9 days ago

80f66afd

[Core] Demo implementation of extensible kv cache memory

zhuohan123 committed 9 days ago

1fa4c3ad

[Bugfix] Don't read KV cache past `seq_len` in triton paged attn kernels (#47305)

njhill committed 9 days ago

Verified 4787f2dd

[ModelRunner V2] Warmup cross-attn properly in encoder-decoder case (#47308)

njhill committed 9 days ago

Verified 8cfeb84d

[ROCm][P/D] MoRIIO toy proxy: support JSON Content-Type for OpenAI clients. (#46482)

lcskrishna committed 9 days ago

Verified 5fd44218

[Bugfix] Prevent padding placeholders from reaching embeddings (#47029)

qianlihuang committed 9 days ago

Verified 00eb7cef

[Bench][BugFix] Fix empty decoder prompt for Cohere ASR in throughput benchmark (#47135)

mganczarenko committed 9 days ago

Verified c8bdcc01

[Spec Decode] DSpark (#46995)

benchislett committed 9 days ago

Verified f5a8d733

[Bugfix] Fix GraniteMoeShared weight loading broken by #41184 (#47031)

mganczarenko committed 9 days ago

Verified 63fcce4d

[Rust Frontend] Split engine core DTOs into separate modules (#47265)

BugenZhao committed 9 days ago

Verified c638f921

[xpu][lora]: Align LoRA implementation with Punica GPU: fix _apply_expand rank mismatch, add_inputs hardcode, and MoE EP (#45368)

chaojun-zhang committed 9 days ago

Verified 13c49f98

[CI] Fix segfault in tracing test (#47299)

njhill committed 9 days ago

Verified f1cf6b00

Migrate GPTBigCode and Starcoder2 to the Transformers modeling backend (#30966)

hmellor committed 9 days ago

Verified a78c1561

docs(security): document gRPC interface as insecure for private use only (#45903)

jperezdealgaba committed 9 days ago

Verified 5c4db60f

[ROCm][MiniMax-M3] Cross-layer lightning-indexer top-k sharing (#47269)

Fangzhou-Ai committed 10 days ago

Verified 4e5ca89c

[Model] Remove AyaVision, MusicFlamingo (#47263)

hmellor committed 10 days ago

Verified a22e0dfc

[Model] Support Hy3 token suffix and JSON Schema array types (#47192)

stevenkuang-tencent committed 10 days ago

Verified cc56379e

[Bugfix] Expose usage field in GenerateResponse for disaggregated serving (#42748)

AIvashov committed 10 days ago

Verified 024b06b0

[CI] Fix various failures on `main` (#47197)

hmellor committed 10 days ago

Verified e7d0fcbc

[ROCm][Perf][Bugfix] DSv4 indexer: use platform FP8 dtype (fnuz) for Q-quant on gfx942 (#46730)

akii96 committed 10 days ago

Verified aa8bb556

[Bugfix] Fix pooled Whisper sliding-window KV sizing (#47071)

andylolu2 committed 10 days ago

Verified fa4bec90

[Test] Run SageMaker handler-override tests in-process via TestClient (#47250)

Jyothirmaikottu committed 10 days ago

Verified dee5da1d

[ROCm][DSV4] Use aiter mHC pre/post as the default ROCm path (#43950)

Fangzhou-Ai committed 10 days ago

Verified ed41aa27

Weight sync refactor + move sparse nccl engine (#44353)

hao-aaron committed 10 days ago

Verified 77a9c5ae

[XPU][UT]Enable ut qk_norm_rope_fusion (#42486)

Yejing-Lai committed 10 days ago

Verified f651a8a9

[CI/Build] Fix LoRA testing (#47242)

jeejeelee committed 10 days ago

Verified 8f82be57

[Core] Make sleep-mode backend capability flags communicator-agnostic (#47243)

matteso1 committed 10 days ago

Verified a461070d

Remove mantis (#46806)

xianbaoqian committed 10 days ago

Verified 4470ae84

[Bugfix] Fix beam search candidate indexing when logprobs count varies (#47126)

chaunceyjiang committed 10 days ago

Verified 697c34b9

Older