vllm-project/vllm

Pull Requests Commits

[ROCm][Build] Reland #62475: paged attention kernel split with `--offload-compress` => x4-x7 wheel build speedup on ROCm (#63105)

fxmarty-amd committed 8 minutes ago

Verified cc1a6d31

[Frontend] Expose --sleep-mode-backend on the CLI (#63138)

Mramaran committed 12 minutes ago

Verified d0eb5d2f

[Bugfix][ROCm][DSv4.1] Gather Engram hashes across DP replicas in the AMD model (#63007)

AndreasKaratzas committed 24 minutes ago

Verified 6d1909a1

[ROCm][DSv4.1] Allocate offloaded Engram tables at their exact size (#63009)

AndreasKaratzas committed 24 minutes ago

Verified f01a76f9

[Bugfix][Model] Build the skipped DeepSeek-VL2 vision tower without meta .item() (#63192)

aarushjain29 committed 46 minutes ago

Verified c0c02786

[Startup][UX] Start multiproc workers concurrently when using spawn (#60028)

kfirtoledo committed 59 minutes ago

Verified 52f44839

upgrade tpu-inference to v0.32.0 and torch to 2.14.0 (#63080)

jayyen01 committed 1 hour ago

Verified 6ee316ea

[Model] Add Recursive Flint (#62938)

WoosukKwon committed 1 hour ago

Verified 899a1689

[Spec Decode][MRV2] Skip drafting in steps where no request can use the drafts (#62784)

zixi-qi committed 1 hour ago

Verified 856b9841

[Bugfix][Anthropic] Map vLLM error types to Anthropic error.type vocabulary (#62294)

zyulai committed 2 hours ago

Verified f0abb2f8

[Core] Add XPress speculative decoding method (#54448)

ZKBig committed 2 hours ago

Verified ce9467b7

[Bugfix][Frontend] Accept and echo prompt-cache and safety_identifier fields in the Responses API (#62856)

albertoperdomo2 committed 2 hours ago

Verified 47c091b1

[Bugfix][Frontend][Kimi K3] Fix Kimi K3 XTML marker leakage in responses API (#62507)

cjackal committed 3 hours ago

Verified 2de0531d

[Quantization] Bump humming-kernels to 0.1.18 and remove DeviceInfo workaround (#62816)

jinzhen-lin committed 3 hours ago

Verified b4411f58

[Docs][KV Offload] Document canonical_layout in the KV offloading usage guide (#61822)

liranschour committed 5 hours ago

Verified c722f3a5

[Bugfix][Attention] Fix FA4 FP8 descale strides for singleton KV heads (#58304)

taeyangyun committed 5 hours ago

Verified 2a038794

[ROCm][CI] Fix int8 GEMM kernel selection logic for ROCm platform (#62099)

qli88 committed 5 hours ago

Verified 1d3e4006

[Bugfix] Add SM100 support for Step-5-Preview (#63147)

jeejeelee committed 5 hours ago

Verified 4a40c611

[bug-fix] Fix KV block leak with fault tolerance enabled in P/D disaggregation (#56430)

tiankaijin committed 5 hours ago

Verified e7bb7e2f

[Frontend] Use VLLMValidationError for client input errors in renderers base (#62738)

MetehanAydemir committed 5 hours ago

Verified 1cc332ee

[Feature] [Frontend] [Reasoning] Support Prompt Embeds inputs for reasoning traces (#62765)

qthequartermasterman committed 5 hours ago

Verified 60a7d568

[Perf][GLM-5.3] Enable GLM low-latency BF16 GEMM on SM107 (#62843)

csahithi committed 5 hours ago

Verified 38955340

[Quantization][Quark] Support MXFP8 (W8A8) routed experts (#63125)

fxmarty-amd committed 5 hours ago

Verified b6031d71

[Benchmark] benchmark_moe: read Gemma 4's MoE shape from its text config (#62526)

Blaze-DSP committed 6 hours ago

Verified b689feb9

[Bugfix][RL] Fix cross-stream reuse of packed NCCL receive buffers (#62555)

dharak-cohere committed 7 hours ago

Verified c723f6d3

[Attention] Decode FlashInfer spec queries with fa2 on pre-SM90 GPUs (#60532)

lesj0610 committed 7 hours ago

Verified 71e93206

[Refactor] Migrate FlashInfer NVLink one-sided to MoEEpCommunication (#61655)

suiyoubi committed 8 hours ago

Verified 58c903ac

[Model] NemotronHMTP: share the target's embedding and LM head (#62194)

0z5a committed 8 hours ago

Verified f235b89b

[CI][Test][1/N] Enable unit tests to run on Rubin (#60298)

johncalesp committed 8 hours ago

Verified 66785950

[ROCm][CI] Download quick-reduce e2e model when cached snapshot has no weights (#63127)

stefankoncarevic committed 8 hours ago

Verified 90aa823c

Older