vllm-project/vllm

Pull Requests Commits

tlrmchlsmth committed 1 year ago

1244c259

minimize changes to gpu_model_runner.py

tlrmchlsmth committed 1 year ago

34fb0cbb

Remove unneeded changes

tlrmchlsmth committed 1 year ago

f33ec6d2

simplify - get rid of tokenshape

tlrmchlsmth committed 1 year ago

c33aeecf

Updates for FA3 and other changes

tlrmchlsmth committed 1 year ago

230730c3

Merge branch 'main' into hacking_full_cudagraph

tlrmchlsmth committed 1 year ago

d151b63b

[Model]: Add `transformers` backend support (#11330)

ArthurZucker committed 1 year ago

Verified a1a2aaad

[ci/build] fix gh200 test (#12681)

youkaichao committed 1 year ago

Verified 1298a400

[cuda] manually import the correct pynvml module (#12679)

youkaichao committed 1 year ago

Verified ad4a9dc8

Fix for attention layers to remain unquantized during moe_wn16 quant (#12570)

srikanthsrnvs committed 1 year ago

Verified b9986454

Properly check if all fused layers are in the list of targets (#12666)

eldarkurtic committed 1 year ago

Verified c5932e5d

make sure mistral_common not imported for non-mistral models (#12669)

youkaichao committed 1 year ago

Verified 20579c0f

[Kernel] port sgl moe_align_block_size kernels (#12574)

chenyang78 committed 1 year ago

Verified 95460fc5

[Doc] Deprecate Discord (#12668)

zhuohan123 committed 1 year ago

Verified 326fcc8b

[doc][misc] clarify VLLM_HOST_IP for multi-node inference (#12667)

youkaichao committed 1 year ago

Verified e6433091

[Misc] Add SPDX-License-Identifier headers to python source files (#12628)

russellb committed 1 year ago

Verified e489ad7a

[Hardware][Intel GPU] add XPU bf16 support (#12392)

jikunshang committed 1 year ago

Verified f256ebe4

[Core][v1] Unify allocating slots in prefill and decode in KV cache manager (#12608)

ShawnD200 committed 1 year ago

Verified f8ece6e1

[V1][Minor] Avoid frequently creating ConstantList (#12653)

WoosukKwon committed 1 year ago

Verified abfcdcdf

[Core] Silence unnecessary deprecation warnings (#12620)

russellb committed 1 year ago

Verified e497f334

[Bugfix] fix moe_wna16 get_quant_method (#12648)

jinzhen-lin committed 1 year ago

Verified baaa2b24

doc: fixing minor typo in readme.md (#12643)

vicenteherrera committed 1 year ago

Verified b4e5c033

Apply torch.compile to fused_moe/grouped_topk (#12637)

mgoin committed 1 year ago

Verified 3194039c

Disable chunked prefill and/or prefix caching when MLA is enabled (#12642)

simon-mo committed 1 year ago

Verified 4f4d427a

[CI/Build] Add label automation for structured-output, speculative-decoding, v1 (#12280)

russellb committed 1 year ago

Verified 1e369839

[Attention] Deepseek v3 MLA support with FP8 compute (#12601)

LucasWilkinson committed 1 year ago

Verified baeded25

Fix: Respect `sparsity_config.ignore` in Cutlass Integration (#12517)

rahul-tuli committed 1 year ago

Verified 3e1c76cf

[Bugfix/CI] Fixup benchmark_moe.py (#12562)

tlrmchlsmth committed 1 year ago

Verified cfa134d2

[ci] Upgrade transformers to 4.48.2 in CI dependencies (#12599)

khluu committed 1 year ago

Verified 35b7a055

Fix target matching for fused layers with compressed-tensors (#12617)

eldarkurtic committed 1 year ago

Verified 1867c258

Older