vllm-project/vllm

Pull Requests Commits

[Bugfix] Use a different prompt for benchmark_serving.py test prompt

tlrmchlsmth committed 1 year ago

221118dc

[MISC] fix typo (#18305)

andyxning committed 1 year ago

Verified 66e63e86

[Model] use AutoWeightsLoader for solar (#18113)

lengrongfu committed 1 year ago

Verified 9214e606

Fixed build on ppc64le due to openssl conflicts (#18262)

npanpaliya committed 1 year ago

Verified f880d425

Update Dockerfile to build for Blackwell (#18095)

mgoin committed 1 year ago

Verified dcfe9523

[Hardware][TPU] Optionally import for TPU backend (#18269)

lsy323 committed 1 year ago

Verified 48ac2bed

[P/D][V1] Support dynamic loading of external KV connector implementations (#18142)

sdavidbd committed 1 year ago

Verified 3e0d4350

[BugFix] Correct max_model_len derivation from config.json for Mistral format (#17937)

princepride committed 1 year ago

Verified 4ee4826e

[Misc] reformat the collect-env output (#18285)

reidliu41 committed 1 year ago

Verified 60017dc8

Move cli args docs to its own page (#18228) (#18264)

strangiato committed 1 year ago

Verified 55f1a468

[V1][P/D] Local attention optimization for NIXL (#18170)

mgoin committed 1 year ago

Verified fd195b19

[Spec Decode] Don't fall back to V0 when spec decoding is enabled (#18265)

WoosukKwon committed 1 year ago

Verified fabe89bb

[Bugfix] fix `an illegal memory access was encountered` of marlin kernel + act_order (#18245)

jinzhen-lin committed 1 year ago

Verified e73b7dfd

[Sampler] Adapt to FlashInfer 0.2.3 sampler API (#15777)

abmfy committed 1 year ago

Verified 7fdfa015

[CI] Assign reviewer to mergify with changes to Tensorizer files (#18278)

sangstar committed 1 year ago

Verified aef94c6d

[BugFix] Fix ordering of KVConnector finished send/rcv sets (#18211)

njhill committed 1 year ago

Verified 0ceaebf8

[BugFix] Fix multi async save in MultiConnector (#18246)

njhill committed 1 year ago

Verified 1db4f47f

[Misc][MacOS] fix bfloat16 error (#18249)

reidliu41 committed 1 year ago

Verified d3d91b6f

[Model] Use autoweightloader for dbrx (#18251)

learner0810 committed 1 year ago

Verified 87d87147

[Fix] Fix typo in `resolve_hf_chat_template` (#18259)

fxmarty-amd committed 1 year ago

Verified a5f8c111

use ceil_div in cutlass block scaling shape check (#17918)

IwakuraRein committed 1 year ago

Verified e23564cb

[Misc] Consolidate Audio tests into multimodal common generation tests (#18214)

Isotr0py committed 1 year ago

Verified 390ec889

[Misc] Add Ray Prometheus logger to V1 (#17925)

eicherseiji committed 1 year ago

Verified 54181767

[PERF] Speed up Qwen2.5-VL model by speed up rotary position embedding (#17973)

vadiklyutiy committed 1 year ago

Verified 67da5720

[doc] fix multimodal example script (#18089)

davidxia committed 1 year ago

Verified 5c04bb8b

[Feature] Support Pipeline Parallism in torchrun SPMD offline inference for V1 (#17827)

luccafong committed 1 year ago

Verified 3d2779c2

Throw better error for when running into k8s service discovery issue (#18209)

wseaton committed 1 year ago

Verified 6b31c84a

Allow users to pass arbitrary JSON keys from CLI (#18208)

hmellor committed 1 year ago

Verified b18201fe

[Model] vLLM v1 supports Medusa (#17956)

skylee-01 committed 1 year ago

Verified f4937a51

[Bugfix][ROCm] Use `chunked_prefill_paged_decode` as fallback for V1 attention on ROCm (#18093)

kliuae committed 1 year ago

Verified ee659e3b

Older