vllm-project/vllm

Pull Requests Commits

[Quantization/NVFP4] Speed up TRTLLM NVFP4 MOE weight loading and fix K/V scale loading for MLA Attn (#25968)

pavanimajety committed 1 year ago

920db411

Fix V1 engine serialization error with Ray distributed executor (#26148)

nrghosh committed 1 year ago

9ea82ecd

Avoid division by zero in cache DS MLA kernel (#26174)

MatthewBonanni committed 1 year ago

13e211bb

Stop mergify from keeping stale PRs alive (#26169)

hmellor committed 1 year ago

2d68bba3

[BugFix][QWEN-VL]fix wrong apply_rotary_emb_torch selection introduced by #24642 (#26123)

xuechendi committed 1 year ago

e45271b0

Fix undefined symbol: cutlass_moe_mm_sm100 (#26098)

jasl committed 1 year ago

84135b14

[Renderer] Move Processor out of LLMEngine (#26165)

DarkLight1337 committed 1 year ago

611c23b6

[Model] Fixed stream generator for gpt-oss + spec-decoding (#26027)

astralord committed 1 year ago

c40c0d9c

[CI/Build] do not enforce precompilation on tpu ci tests (#25992)

sixiang-google committed 1 year ago

d8b1f9cc

[Model] Supplement to PR 24862: Pass param prefix to LLMHead (#25805)

whx-sjtu committed 1 year ago

fac9b430

[backends][short_conv] CUDA graph piecewise edits (#24215)

paulpak58 committed 1 year ago

c6f384da

[Bugfix] Re-enable prefill of max model length (#24446)

yannicks1 committed 1 year ago

7faf51f1

[Renderer] Move Processor out of AsyncLLM (#24138)

KKSK-DON committed 1 year ago

ff1daf6c

Quick fix for IMA with the Prefix Prefill kernel during graph capture (#25983)

SageMoore committed 1 year ago

f3768686

[Doc] Fixed shape description for fused_batched_moe.py (#25668)

Egor-Krivov committed 1 year ago

564233d5

[Multi Modal] Configurable MM Profiling (#25631)

wwl2755 committed 1 year ago

2bcc7450

[openai] Fix missing tool usage check (system message) (#24768)

levunet committed 1 year ago

fa29d31f

[NIXL][Misc] Expose metrics from NIXL for logging to CLI (#25388)

NickLucche committed 1 year ago

2168fc8f

[CI] Fix distributed hybrid tests in CI (#26155)

tdoublep committed 1 year ago

8d332b3c

[test utils] correct wrong typing (#26159)

yannicks1 committed 1 year ago

c6344152

[Model] Use `merge_by_field_config` for MM models (InternVL family) (#26153)

DarkLight1337 committed 1 year ago

c81dc099

add(v1): RequestStatesStats to RequestOutput (#24947)

huijjj committed 1 year ago

edaae182

[Perf] Optimize `reshape_and_cache` CUDA Kernel (#25955)

ZJY0516 committed 1 year ago

5b80f220

[Input] Remove unused `prompt` field (#26097)

DarkLight1337 committed 1 year ago

ae03f4c0

[Misc] Remove typing.List (#26150)

varun-sundar-rabindranath committed 1 year ago

7e4b1861

[BUG] Reorder model config creation (#26124)

hao-aaron committed 1 year ago

d628fa1e

`FusedMoE` support for the Transformers backend (#22650)

hmellor committed 1 year ago

6b12b2ee

[Model] Use `merge_by_field_config` for MM models (G) (#26117)

DarkLight1337 committed 1 year ago

bbeace23

[Bugfix] Fix import `gemm_afp4wfp4` failure on AMD (#26068)

zhewenl committed 1 year ago

09b1a567

[ROCm] [VL] [Bugfix] Fix vit flash attn dispatcher logic for ROCm (#26104)

tjtanaa committed 1 year ago

f35f896e

Older