vllm-project/vllm

Pull Requests Commits

[BugFix] Pass in correct VLLM config in FlashInfer backend (#13207) (#16973)

timzsu committed 233 days ago

Verified f961d7f6

Improve configs - `SpeculativeConfig` (#16971)

hmellor committed 233 days ago

Verified d0591104

[Bugfix] Fix distributed bug again in Qwen2.5-VL & Qwen2.5-Omni (#16974)

fyabc committed 233 days ago

Verified 571e8dd6

[Misc] refactor example series (#16972)

reidliu41 committed 233 days ago

Verified 4b91c927

[FEAT][ROCm] Integrate Paged Attention Kernel from AITER (#15001)

vllmellm committed 233 days ago

Verified 0e237f00

[Doc] Improve documentation for multimodal CLI args (#16960)

DarkLight1337 committed 233 days ago

Verified 8f7bace7

[BugFix] Fix incremental detokenization perf issue (#16963)

njhill committed 233 days ago

Verified e4d61442

[Kernel] Support Microsoft Runtime Kernel Lib for our Low Precision Computation - BitBLAS (#6036)

LeiWang1999 committed 233 days ago

Verified 8d32dc60

[V1] Remove pre-allocation for KV cache (#16941)

WoosukKwon committed 233 days ago

Verified c4ab9f3e

[Model] Use autoweightloader for mamba (#16950)

sfeng33 committed 233 days ago

Verified 2689d5c0

[Bugfix] Fix the issue where llm.generate cannot be called repeatedly after setting GuidedDecodingParams (#16767)

chaunceyjiang committed 233 days ago

Verified acba33a0

[Perf] Optimize `_update_states` for GPU model runner (#16910)

SnowCharmQ committed 233 days ago

Verified a114bf20

[Doc] Update ai_accelerator/hpu-gaudi.inc.md (#16956)

windsonsea committed 233 days ago

Verified 3097ce3a

[Bugfix] Fix f-string for Python 3.9-3.11 (#16962)

DarkLight1337 committed 233 days ago

Verified d6da9322

Support S3 Sharded loading with RunAI Model Streamer (#16317)

omer-dayan committed 233 days ago

Verified 71ce4404

[Performance][ROCm] Add skinny gemms for unquantized linear on ROCm (#15830)

charlifu committed 234 days ago

Verified 188b7f9b

[V1] Remove additional_config check (#16710)

wangxiyuan committed 234 days ago

Verified b9b47469

[Kernel] Add expert_map support to Cutlass FP8 MOE (#16861)

varun-sundar-rabindranath committed 234 days ago

Verified 7b8a2ab7

[Misc] Remove the chunked prefill warning for LoRA (#16925)

jeejeelee committed 234 days ago

Verified c9acbf11

[ROCm] Add aiter tkw1 kernel for Llama4 fp8 (#16727)

kliuae committed 234 days ago

Verified 5b794cae

[Bugfix]: fix issue with n>1 sampling on v1 requests overriding each other (#16863)

jeffrey-dot-li committed 234 days ago

Verified 0e425449

[BugFix][Spec Decode] No in-place update to draft probs (#16952)

WoosukKwon committed 234 days ago

Verified 1311913f

[Doc] Remove unnecessary V1 flag (#16924)

DarkLight1337 committed 234 days ago

Verified 29f395c9

[TPU][V1] Enable Top-P (#16843)

NickLucche committed 234 days ago

Verified fa3bba2a

[V1] V1 FlashInfer Attention (#16684)

mgoin committed 234 days ago

Verified 986537f1

[TPU][V1] Capture multimodal encoder during model compilation (#15051)

NickLucche committed 234 days ago

Verified 21020752

Update Qwen1.5-MoE-W4A16-compressed-tensors.yaml (#16946)

mgoin committed 234 days ago

Verified 71eda0bb

[TPU][V1] Implicitly adjust page size when there's SMEM OOM (#16871)

yaochengji committed 234 days ago

Verified 471fe656

[V1][Spec Decode] Handle draft tokens beyond max_model_len (#16087)

WoosukKwon committed 234 days ago

Verified 3a0fba5c

[Core] Speed up decode by remove synchronizing operation in sampler (#16436)

chanh committed 234 days ago

Verified 299ebb62

Newer Older