Commits vllm-project/vllm

[Model Runner V2] Order uniform decodes first in the input batch

WoosukKwon committed 4 hours ago

57c87028

[MoE] Plumb gemm1_alpha/beta/clamp_limit into TRT-LLM FP8 MoE (#45723)

zyongye committed 11 hours ago

Verified fa248139

[DSV4] Better MXFP8 quantization kernel (#47229)

zyongye committed 11 hours ago

Verified d3229431

[Bugfix] Don't read KV cache past `seq_len` in triton paged attn kernels (#47305)

njhill committed 12 hours ago

Verified 4787f2dd

[ModelRunner V2] Warmup cross-attn properly in encoder-decoder case (#47308)

njhill committed 13 hours ago

Verified 8cfeb84d

[ROCm][P/D] MoRIIO toy proxy: support JSON Content-Type for OpenAI clients. (#46482)

lcskrishna committed 13 hours ago

Verified 5fd44218

[Bugfix] Prevent padding placeholders from reaching embeddings (#47029)

qianlihuang committed 16 hours ago

Verified 00eb7cef

[Bench][BugFix] Fix empty decoder prompt for Cohere ASR in throughput benchmark (#47135)

mganczarenko committed 16 hours ago

Verified c8bdcc01

[Spec Decode] DSpark (#46995)

benchislett committed 17 hours ago

Verified f5a8d733

[Bugfix] Fix GraniteMoeShared weight loading broken by #41184 (#47031)

mganczarenko committed 17 hours ago

Verified 63fcce4d

[Rust Frontend] Split engine core DTOs into separate modules (#47265)

BugenZhao committed 18 hours ago

Verified c638f921

[xpu][lora]: Align LoRA implementation with Punica GPU: fix _apply_expand rank mismatch, add_inputs hardcode, and MoE EP (#45368)

chaojun-zhang committed 18 hours ago

Verified 13c49f98

[CI] Fix segfault in tracing test (#47299)

njhill committed 18 hours ago

Verified f1cf6b00

Migrate GPTBigCode and Starcoder2 to the Transformers modeling backend (#30966)

hmellor committed 18 hours ago

Verified a78c1561

docs(security): document gRPC interface as insecure for private use only (#45903)

jperezdealgaba committed 19 hours ago

Verified 5c4db60f

[ROCm][MiniMax-M3] Cross-layer lightning-indexer top-k sharing (#47269)

Fangzhou-Ai committed 21 hours ago

Verified 4e5ca89c

[Model] Remove AyaVision, MusicFlamingo (#47263)

hmellor committed 21 hours ago

Verified a22e0dfc

[Model] Support Hy3 token suffix and JSON Schema array types (#47192)

stevenkuang-tencent committed 22 hours ago

Verified cc56379e

[Bugfix] Expose usage field in GenerateResponse for disaggregated serving (#42748)

AIvashov committed 22 hours ago

Verified 024b06b0

[CI] Fix various failures on `main` (#47197)

hmellor committed 23 hours ago

Verified e7d0fcbc

[ROCm][Perf][Bugfix] DSv4 indexer: use platform FP8 dtype (fnuz) for Q-quant on gfx942 (#46730)

akii96 committed 23 hours ago

Verified aa8bb556

[Bugfix] Fix pooled Whisper sliding-window KV sizing (#47071)

andylolu2 committed 23 hours ago

Verified fa4bec90

[Test] Run SageMaker handler-override tests in-process via TestClient (#47250)

Jyothirmaikottu committed 23 hours ago

Verified dee5da1d

[ROCm][DSV4] Use aiter mHC pre/post as the default ROCm path (#43950)

Fangzhou-Ai committed 1 day ago

Verified ed41aa27

Weight sync refactor + move sparse nccl engine (#44353)

hao-aaron committed 1 day ago

Verified 77a9c5ae

[XPU][UT]Enable ut qk_norm_rope_fusion (#42486)

Yejing-Lai committed 1 day ago

Verified f651a8a9

[CI/Build] Fix LoRA testing (#47242)

jeejeelee committed 1 day ago

Verified 8f82be57

[Core] Make sleep-mode backend capability flags communicator-agnostic (#47243)

matteso1 committed 1 day ago

Verified a461070d

Remove mantis (#46806)

xianbaoqian committed 1 day ago

Verified 4470ae84

[Bugfix] Fix beam search candidate indexing when logprobs count varies (#47126)

chaunceyjiang committed 1 day ago

Verified 697c34b9