vllm-project/vllm

Pull Requests Commits

[Attention] MLA - cutlass decode with unresticted num_heads

alexm-redhat committed 1 year ago

32e44816

[Docs] Data Parallel deployment documentation (#20768)

njhill committed 1 year ago

Verified 9907fc44

[Kernel] Basic tuned configs for NVFP4 CUTLASS dense GEMM (#20646)

mgoin committed 1 year ago

Verified d47661f0

[Misc] Add unit tests for MoE ModularKernel combinations + Profiling utility (#20449)

varun-sundar-rabindranath committed 1 year ago

Verified 53fa4573

[doc] fix ordered list issue (#20819)

reidliu41 committed 1 year ago

Verified 6fb16244

[Bugfix] Add missing field to TritonLanguagePlaceholder (#20812)

bigPYJ1151 committed 1 year ago

Verified 66177189

Temporarily suspend google/gemma-3-1b-it. (#20722)

QiliangCui committed 1 year ago

Verified b4f0b5f9

[Bugfix] Refactor `/invocations` to be task-agnostic (#20764)

DarkLight1337 committed 1 year ago

Verified cbd14ed5

[Core] Add Flashinfer TRTLLM Backend for Flashinfer decode path (SM100). (#19825)

pavanimajety committed 1 year ago

Verified 7bd4c37a

[Quantization][1/N] MoE support BNB-Inflight Quantization (#20061)

jeejeelee committed 1 year ago

Verified 8020e98c

[Bugfix] Upgrade depyf to 0.19 and streamline custom pass logging (#20777)

ProExpertProg committed 1 year ago

Verified 762be26a

[doc] fold long code block (#20795)

reidliu41 committed 1 year ago

Verified 6a9e6b2a

[V1] Enable Mamba2 layers other than MambaMixer2 in the v1 engine (#20660)

nopperl committed 1 year ago

Verified 5d09152f

[Perf][fp8] Use CustomOp abstraction for fp8 quant for better perf (#19830)

ProExpertProg committed 1 year ago

Verified 31d5c179

[XPU] XCCL support enabled in torch 2.8.0.dev nightly builds (#20705)

ratnampa committed 1 year ago

Verified 35514b68

[Feature] Integrate SM100 DeepGEMM support (#20087)

yewentao256 committed 1 year ago

Verified e2de455c

[Attention] MLA - Flashinfer Ragged Prefill (#20034)

alexm-redhat committed 1 year ago

Verified 5b032352

[Model] Support HF format of minimax (#20211)

mgoin committed 1 year ago

Verified 922f3164

[fix]: disable cutlass block scaled group gemm for EP (#20781)

djmmoss committed 1 year ago

Verified 5923ab95

Add kimi-k2 tool parser (#20789)

MoyanZitto committed 1 year ago

Verified 0cf893ca

[CI Bugfix] Specify same TORCH_CUDA_ARCH_LIST for flashinfer aot and install (#20772)

mgoin committed 1 year ago

Verified cf75cd20

[Docs] Lazy import gguf (#20785)

simon-mo committed 1 year ago

Verified b854321f

[Bugfix][Benchmark] Make sure the output length > 0 when testing prefill workload. (#20786)

KuntaiDu committed 1 year ago

Verified 5b6fe23d

[Misc] MoE ModularKernel : Introduce TopKWeightAndReduce (#20648)

varun-sundar-rabindranath committed 1 year ago

Verified f0c98cae

[KVConnector] Always call connector `clear_metadata()` at end of step (#20756)

njhill committed 1 year ago

Verified 574ad60d

[Bugfix] Fused MoE Modular Kernel chunking loop (#20392)

varun-sundar-rabindranath committed 1 year ago

Verified fdadb6f4

[Core] Add Support for Default Modality Specific LoRAs [generate / chat completions] (#19126)

alex-jw-brooks committed 1 year ago

Verified 41060c6e

[Bugfix] Remove assertion of expert_map being None (#20714)

minosfuture committed 1 year ago

Verified 3de2ed76

[CI] Fix pre commit issue (#20782)

yewentao256 committed 1 year ago

Verified 299252ea

[V0][V1][Core] Add outlines integration for V1, and update V0 integration. (#15975)

unaidedelf8777 committed 1 year ago

Verified d6902ce7

Older