vllm-project/vllm

Pull Requests Commits

[Bugfix] Add MigCudaPlatform to avoid CUDA init before fork

khluu committed 165 days ago

5d236e5c

khluu committed 166 days ago

f6832665

[Bugfix] Disable expandable_segments on MIG to fix PyTorch NVML assert

khluu committed 166 days ago

8dde2576

[Bugfix] Use NonNvmlCudaPlatform on MIG devices to fix NVML masking OOM errors

khluu committed 167 days ago

ae60bf5e

[Bugfix] Respect VLLM_WEIGHT_OFFLOADING_DISABLE_PIN_MEMORY in prefetch offloader (#37699)

he-yufeng committed 167 days ago

Verified bcc2306c

[Test] Refactor hard coded device string in test files under compile/quantization/models/model_executor folders (#38901)

wincent8 committed 167 days ago

Verified 3abf8584

[Attention Backend] TurboQuant: 2-bit KV cache compression with 4x capacity (#38479)

vibhavagarwal5 committed 167 days ago

Verified f4b42df0

[Model Runner V2] Disable piecewise cudagraph mode fallback for eagle draft decodes (#39773)

TheEpicDolphin committed 167 days ago

Verified 3bfe55a0

[CI] Add PyTorch nightly build and test pipeline (#37226)

atalman committed 167 days ago

Verified b569620f

[Bugfix] Disable FlashInfer CUTLASS MoE on SM121 (DGX Spark) (#39825)

mgoin committed 167 days ago

Verified 65b98089

[Hybrid] Simplify accepted token counting in spec decode for hybrid models (#38372)

fuscof-ibm committed 167 days ago

Verified 507df79a

[Bugfix][Mooncake] Fix thread-local CUDA context for NVLink transfers in _send_blocks (#39548)

zhewenl committed 167 days ago

Verified 1696c864

[Bug] Fix batch invariance nvfp4 support (#39820)

yewentao256 committed 167 days ago

Verified 2ad10292

fix(lmcache): correct store for cached requests while enable prefix cache (#39719)

maobaolong committed 167 days ago

Verified b2f749dc

[Reasoning][Frontend] Add model config to adjust_request in reasoning parser (#37848)

rishitdholakia13 committed 167 days ago

Verified 70ed0155

[MoE Refactor] Refactor ZeroExpertFusedMoE into new framework (#35549)

bnellnm committed 167 days ago

Verified 19ec9a0a

[MoE] Move GPT OSS Triton kernel experts into fused_moe/experts/ (#39007)

Jackmin801 committed 167 days ago

Verified 1a9353bb

[Mamba] Flashinfer selective_state_update (#36162)

roikoren755 committed 167 days ago

Verified ecf5ff7c

[CI][KVConnector][Metrics] Update multi KV connector edge case according to prefill stats changes (#39808)

ZhanqiuHu committed 167 days ago

Verified 30679319

[Kernel] Support TRTLLM GEN NVFP4 MoE for non-512-aligned hidden dims via weight padding (#39510)

danielafrimi committed 167 days ago

Verified 240f2636

add warning when FP8 KV cache misses prefill query quantization (#39752)

qiching committed 167 days ago

Verified dc8df110

[KV Offload] Unified memory layout for offloading workers (#37206)

omerpaz95 committed 167 days ago

Verified be0c855e

[ROCm] Align AiterFlashAttentionImpl attn_type check with backend (#39119)

Bortlesboat committed 167 days ago

Verified e64b39ea

[compile] Nest inductor cache under AOT compile dir (#39718)

fulvius31 committed 168 days ago

Verified 2faad083

[Bugfix][ROCm]: Allow `gpt_oss_mxfp4` quantization method on rocm (#39754)

Rohan138 committed 168 days ago

Verified 23f37602

[Core][Metrics] Remove unused `SchedulerStats.encoder_cache_usage` (#39693)

markmc committed 168 days ago

Verified 906a8c15

[ROCm][CI] Fix condition for `test_per_token_group_quant_fp8_packed` (#39730)

micah-wil committed 168 days ago

Verified 4f4f8eaa

Bugfix: `use_existing_torch.py`: Glob recursive subdirs in requirements (fixes #39024) (#39793)

netanel-haber committed 168 days ago

Verified b6890a12

Measure encoder compile time seperate from llm backbone (#39240)

Lucaskabela committed 168 days ago

Verified c08f3b2a

[PluggableLayer][3/N] Apply PluggableLayer to moe-related layers. (#33556)

whx-sjtu committed 168 days ago

Verified f02b3269

Older