vllm-project/vllm

Pull Requests Commits

[XPU] properly handle q_descale on XPU as quant query input not supported (#39676)

yma11 committed 79 days ago

Verified 3beb57a2

FIX: support language_model.backbone naming in NemotronH Nano VL quantization config (#39901)

danielafrimi committed 79 days ago

Verified 8b553193

[BugFix][Graph] fix: handle empty sym_shape_indices in PiecewiseBackend. (#39395)

chaunceyjiang committed 79 days ago

Verified db8d4a4a

[XPU][MXFP4] add mxfp4 quant op for XPU (#39857)

zufangzhu committed 80 days ago

Verified fc701c80

[Metrics] Add request_id to FinishedRequestStats to enable correlation between metrics and requests (#39710)

Csrayz committed 80 days ago

Verified 68be0f85

[Quantization][Autoround][CPU] Add W4A16 Support (#38192)

Zhenzhong1 committed 80 days ago

Verified 60995c05

fix online fp8 for MiniCPM models (#39862)

yma11 committed 80 days ago

Verified 29e5d102

[kv_offload+HMA][3/N]: Remove block_size from KVEvents (#36644)

orozery committed 80 days ago

Verified 235e1f93

[Bugfix] Fix tool_calls Iterable consumed when debug logging is enabled (#34844)

wojciech-wais committed 80 days ago

Verified 431cea3e

[CI][NIXL] Fix PD CI breakage: pin nixl-cu{12,13} versions (#39851)

ZhanqiuHu committed 80 days ago

Verified 799973af

[Bugfix] Respect VLLM_WEIGHT_OFFLOADING_DISABLE_PIN_MEMORY in prefetch offloader (#37699)

he-yufeng committed 80 days ago

Verified bcc2306c

[Test] Refactor hard coded device string in test files under compile/quantization/models/model_executor folders (#38901)

wincent8 committed 80 days ago

Verified 3abf8584

[Attention Backend] TurboQuant: 2-bit KV cache compression with 4x capacity (#38479)

vibhavagarwal5 committed 80 days ago

Verified f4b42df0

[Model Runner V2] Disable piecewise cudagraph mode fallback for eagle draft decodes (#39773)

TheEpicDolphin committed 80 days ago

Verified 3bfe55a0

[CI] Add PyTorch nightly build and test pipeline (#37226)

atalman committed 80 days ago

Verified b569620f

[Bugfix] Disable FlashInfer CUTLASS MoE on SM121 (DGX Spark) (#39825)

mgoin committed 80 days ago

Verified 65b98089

[Hybrid] Simplify accepted token counting in spec decode for hybrid models (#38372)

fuscof-ibm committed 80 days ago

Verified 507df79a

[Bugfix][Mooncake] Fix thread-local CUDA context for NVLink transfers in _send_blocks (#39548)

zhewenl committed 80 days ago

Verified 1696c864

[Bug] Fix batch invariance nvfp4 support (#39820)

yewentao256 committed 80 days ago

Verified 2ad10292

fix(lmcache): correct store for cached requests while enable prefix cache (#39719)

maobaolong committed 80 days ago

Verified b2f749dc

[Reasoning][Frontend] Add model config to adjust_request in reasoning parser (#37848)

rishitdholakia13 committed 80 days ago

Verified 70ed0155

[MoE Refactor] Refactor ZeroExpertFusedMoE into new framework (#35549)

bnellnm committed 80 days ago

Verified 19ec9a0a

[MoE] Move GPT OSS Triton kernel experts into fused_moe/experts/ (#39007)

Jackmin801 committed 80 days ago

Verified 1a9353bb

[Mamba] Flashinfer selective_state_update (#36162)

roikoren755 committed 80 days ago

Verified ecf5ff7c

[CI][KVConnector][Metrics] Update multi KV connector edge case according to prefill stats changes (#39808)

ZhanqiuHu committed 80 days ago

Verified 30679319

[Kernel] Support TRTLLM GEN NVFP4 MoE for non-512-aligned hidden dims via weight padding (#39510)

danielafrimi committed 80 days ago

Verified 240f2636

add warning when FP8 KV cache misses prefill query quantization (#39752)

qiching committed 80 days ago

Verified dc8df110

[KV Offload] Unified memory layout for offloading workers (#37206)

omerpaz95 committed 80 days ago

Verified be0c855e

[ROCm] Align AiterFlashAttentionImpl attn_type check with backend (#39119)

Bortlesboat committed 80 days ago

Verified e64b39ea

[compile] Nest inductor cache under AOT compile dir (#39718)

fulvius31 committed 80 days ago

Verified 2faad083

Older