vllm-project/vllm

Pull Requests Commits

alexm-redhat committed 1 year ago

0a02744d

add cuda graph support to triton_mla attention

alexm-redhat committed 1 year ago

984ffddd

review comments

LucasWilkinson committed 1 year ago

135c404f

review comments

LucasWilkinson committed 1 year ago

7241acbd

Update vllm/config.py

LucasWilkinson committed 1 year ago

Verified 2b140deb

renaming for consistency

LucasWilkinson committed 1 year ago

2326814c

review comments

LucasWilkinson committed 1 year ago

534cd000

Update vllm/attention/backends/mla/utils.py

LucasWilkinson committed 1 year ago

Verified aa19f297

Update utils.py

LucasWilkinson committed 1 year ago

Verified 4880a43d

LucasWilkinson committed 1 year ago

3895bba8

fix VLLM_MLA_PERFORM_MATRIX_ABSORPTION=0

LucasWilkinson committed 1 year ago

f23d126a

squashed commits

LucasWilkinson committed 1 year ago

ec8c1cf7

[V1][Metrics] Add GPU cache usage % gauge (#12561)

markmc committed 1 year ago

Verified f17f1d46

[Misc][MoE] add Deepseek-V3 moe tuning support (#12558)

divakar-amd committed 1 year ago

Verified 1c1bb0bb

[V1][BugFix] Free encoder cache for aborted requests (#12545)

WoosukKwon committed 1 year ago

Verified e0cc5f25

Revert "[Build/CI] Fix libcuda.so linkage" (#12552)

tlrmchlsmth committed 1 year ago

Verified 73aa6cfd

[Kernel] add triton fused moe kernel for gptq/awq (#12185)

jinzhen-lin committed 1 year ago

Verified 27b78c73

[Hardware][NV] Fix Modelopt model loading for k-v-scales for Llama models. (#11787)

pavanimajety committed 1 year ago

Verified b02fd288

[Frontend] Support override generation config in args (#12409)

liuyanyi committed 1 year ago

Verified ff7424f4

[Model] Refactoring of MiniCPM-V and add MiniCPM-o-2.6 support for vLLM (#12069)

HwwwwwwwH committed 1 year ago

Verified d93bf4da

[Bugfix] handle alignment of arguments in convert_sparse_cross_attention_mask_to_dense (#12347)

tjohnson31415 committed 1 year ago

Verified 036ca94c

Fix the pydantic logging validator (#12420)

maxdebayser committed 1 year ago

Verified ef001d98

[V1] Improve Error Message for Unsupported Config (#12535)

robertgshaw2-redhat committed 1 year ago

Verified 5f671cb4

Bugfix for whisper quantization due to fake k_proj bias (#12524)

mgoin committed 1 year ago

Verified bd02164c

[V1][Metrics] Add TTFT and TPOT histograms (#12530)

markmc committed 1 year ago

Verified 46fb0567

[Doc] Convert docs to use colon fences (#12471)

hmellor committed 1 year ago

Verified dd6a3a02

[Frontend] Support reasoning content for deepseek r1 (#12473)

gaocegege committed 1 year ago

Verified a7e3eba6

[TPU] Add example for profiling TPU inference (#12531)

mgoin committed 1 year ago

Verified fbb5bd4c

[Kernel] Pipe attn_logits_soft_cap through paged attention TPU kernels (#12482)

fenghuizhang committed 1 year ago

Verified 80fcc3ed

[V1][Metrics] Add per-request prompt/generation_tokens histograms (#12516)

markmc committed 1 year ago

Verified c386c43c

Older