vllm-project/vllm

Pull Requests Commits

Merge branch 'main' into fix-mtp-dummy-run-assertion

luccafong committed 92 days ago

Verified 43bd060b

[Perf] Improve default triton fused moe configs (#34846)

mgoin committed 92 days ago

Verified 22a97e66

[RL] Validation for pause_mode='keep' (#34992)

hao-aaron committed 92 days ago

Verified 596ed1f0

[Misc] Monitor interface changes (#35113)

NickLucche committed 92 days ago

Verified b8d8b7e9

Enforce that `model` is the first positional arg when `--served-model-name` is used (#34973)

hmellor committed 92 days ago

Verified 28c5e69b

Fix custom processors that use deleted import for Transformers v5 (#35101)

hmellor committed 92 days ago

Verified 864167d3

[Bugfix] Fix prefix caching for Mamba 'all' mode (Nemotron models) (#34874)

haosdent committed 92 days ago

Verified a2ba6a52

Use Xet high performance mode for Transformers v5 (#35098)

hmellor committed 92 days ago

Verified c4f38696

[Bugfix] Fix MRotaryEmbedding missing `truncate` attr with YaRN scaling (#35080)

haosdent committed 92 days ago

Verified a7f341c3

[CI] Skip Responses API (#34990)

robertgshaw2-redhat committed 92 days ago

Verified d13ece38

[Metrics] Add Prometheus counters for Model FLOPs Utilization (MFU) (#30950)

markmc committed 92 days ago

Verified 5cc7c445

[kv-cache, ct] Use compressed-tensors as a source of ground-truth for quant strategies (#34254)

eldarkurtic committed 92 days ago

Verified b95bb692

[Refactor] Decouple TimingContext from InputProcessingContext (#35083)

DarkLight1337 committed 92 days ago

Verified 39264545

[Llama4,CI] Bring back Llama-4 bug fixes, and also fix Maverick tests (#35033)

eldarkurtic committed 92 days ago

Verified 1e8438a8

[ModelBash][DSV3] Add TRTLLM DSV3 Router GEMM kernel (6% B1 Speedup) (#34302)

robertgshaw2-redhat committed 92 days ago

Verified 8435b2e0

[XPU] allow TORCH_SDPA/TRITON_ATTN as XPU vit Backend (#35010)

yma11 committed 92 days ago

Verified b1b5e045

[ROCm][CI] Fix spec decode profile assertion and logprob test determinism (#35043)

AndreasKaratzas committed 92 days ago

Verified 5f68464f

[CLEANING] Remove unused disable_by_batch_size from SpeculativeConfig (#35060)

VincentG1234 committed 92 days ago

Verified aa08a30f

[Refactor] Remove dead private func `_fp8_perm` and `_extract_mask_for_item` (#35068)

yewentao256 committed 92 days ago

Verified 7f40e9e5

Fix pipeline parallel with embed scaling in the Transformers modelling backend (#35094)

hmellor committed 92 days ago

Verified 103e614b

[Feature] Lazy import for the "mistral" tokenizer module. (#34651)

nascheme committed 93 days ago

Verified 54e2f83d

fix: Apply embedding_multiplier to inputs_embeds (#34813)

gabe-l-hart committed 93 days ago

Verified e631f8e7

[BugFix]: Fix local mypy issues (#34739)

hickeyma committed 93 days ago

Verified e97c46a9

[Bugfix] Fix kernel benchmark (#33752)

jeejeelee committed 93 days ago

Verified 7291d1b2

[Refactor] Simplify dummy data generation (#35025)

DarkLight1337 committed 93 days ago

Verified 987506bc

[Model Runner V2] Remove propose_draft method (#35070)

WoosukKwon committed 93 days ago

Verified c645e9a2

[Model Runner V2][Minor] Remove redundant `do_spec_decode` field (#35039)

njhill committed 93 days ago

Verified 944ffb59

[Spec Decode] Reduce TP communication for speculative decoding draft token generation (#34049)

zixi-qi committed 93 days ago

Verified 2bcf71b9

[Model] Add NVFP4 quantization support for Step3.5-Flash (#34478)

tacos8me committed 93 days ago

Verified b7892a3b

[Bug] Refactor max_num_batched_tokens to account for drafting (#34898)

benchislett committed 93 days ago

Verified 682566b1

Older