vllm-project/vllm

Pull Requests Commits

Sage Moore fixes for full cuda graph support for DeepEP+DeepGEMM LL

tlrmchlsmth committed 1 year ago

e53382cc

refactor example - qwen3_reranker (#19847)

reidliu41 committed 1 year ago

Verified 26d34eb6

[Bugfix][CPU] Fix InputBatch for pooling models in the CPU v1 (#20014)

bigPYJ1151 committed 1 year ago

Verified 53da4cd3

[PERF] Speedup of MRoPE prepare inputs (#19939)

vadiklyutiy committed 1 year ago

Verified 9a3b8832

add some examples for other benchmark scripts (#19893)

reidliu41 committed 1 year ago

Verified 3014c920

[doc] Fix broken link in the installation for CPU (#19980)

yankay committed 1 year ago

Verified 0eed5169

[Misc][Tools][Benchmark] Add profile to autotune script (#19711)

Chenyaaang committed 1 year ago

Verified ee5ad8d2

Update test case parameter to have the throughput above 8.0 (#19994)

QiliangCui committed 1 year ago

Verified a738dbb2

[TPU] Fix tpu model runner test (#19995)

Chenyaaang committed 1 year ago

Verified 33d5e29b

[Bugfix][Benchmark] Fix Marlin benchmark (#19929)

22quinn committed 1 year ago

Verified 4671ac6e

Feat Dynamic Quantization for MoE Layers in GPTQ Marlin Backend (#19395)

Jun-Howie committed 1 year ago

Verified dd2ccf8d

[CI/Build] Push latest tag for cpu and neuron docker image (#19897)

22quinn committed 1 year ago

Verified a3bc76e4

[Feature] Support sequence parallelism for static fp8 quantization (#19181)

cascade812 committed 1 year ago

Verified e6327c9b

[Misc] Add type alias `ReqId` and `EngineId` for better readability (#19880)

lk-chen committed 1 year ago

Verified d0132f02

[Bugfix][v1] Fix step pooler implementation and step pooling usage in v1 (#19956)

Isotr0py committed 1 year ago

Verified 61f4fc5d

[EP+DP] Optimize the little operations in the DeepGEMM + DeepEP low latency case (#19885)

tlrmchlsmth committed 1 year ago

Verified 68aaeb37

[Docs] Fix syntax highlighting of shell commands (#19870)

lgeiger committed 1 year ago

Verified c3649e4f

[doc] improve readability for long commands (#19920)

reidliu41 committed 1 year ago

Verified 53243e5c

[Bugfix] Fix CI bitsandbytes failure (#19969)

jeejeelee committed 1 year ago

Verified a6e6604d

[doc] use MkDocs collapsible blocks - supplement (#19973)

reidliu41 committed 1 year ago

Verified b82e0f82

[Doc] Update V1 status for decoder-only embedding models (#19952)

Isotr0py committed 1 year ago

Verified 5111642a

[BugFix][P/D] Fix for cases where _recving_transfers can be cleaned up when *all* transfer done (#19874)

lk-chen committed 1 year ago

Verified 1bcd15ed

[P/D][NixlConnector] Support `tp_size > num_kv_heads` deployments (#19691)

NickLucche committed 1 year ago

Verified 2ebff5b7

[doc] Fold long code blocks to improve readability (#19926)

reidliu41 committed 1 year ago

Verified f17aec0d

Fix(models/siglip): Add compatibility for Gemma models quantized by llm-compressor (#19643)

Flink-ddd committed 1 year ago

Verified 493c2753

[Misc] Configurable timeout for execute_model RPC calls via env var (#19544)

jinqinn committed 1 year ago

Verified f39ab2d4

[Core] feat: Implement Priority Scheduling in V1 Engine (#19057)

amitm02 committed 1 year ago

Verified 4a0f7888

[Perf][CLI] Improve overall startup time (#19941)

aarnphm committed 1 year ago

Verified c4cf2606

[BugFix] Add an env to disable moe chunking to work around compile incompatibility (#19642)

yeqcharlotte committed 1 year ago

Verified 33d51f59

[Chore] dedup logs (#19955)

aarnphm committed 1 year ago

Verified e91386cd

Older