vllm-project/vllm

Pull Requests Commits

[V1] TPU - Remove self.kv_caches

alexm-redhat committed 1 year ago

bfff9bcd

[V1][Frontend] Add Testing For V1 Runtime Parameters (#14159)

robertgshaw2-redhat committed 1 year ago

Verified 257e200a

Small update for external_launcher backend docs (#14288)

zhe-thoughts committed 1 year ago

Verified 47d4a7e0

[Doc] [3/N] Refer code examples for common cases in dev multimodal processor (#14278)

DarkLight1337 committed 1 year ago

Verified 7f89a594

[Doc] Update nginx guide: remove privileged from vllm container run and add target GPU ID (#14217)

iacolippo committed 1 year ago

Verified 961644e6

[Bugfix][V1] Fix allowed_token_ids for v1 Sampler (#14169)

houseroad committed 1 year ago

Verified 8d6cd32b

[Misc][V1] Avoid using `envs.VLLM_USE_V1` in mm processing (#14256)

ywang96 committed 1 year ago

Verified ec79b67c

[Frontend] Allow return_tokens_as_token_ids to be passed as a request param (#14066)

benchislett committed 1 year ago

Verified 32985bed

Temporarily disable test_awq_gemm_opcheck (#14251)

mgoin committed 1 year ago

Verified dae9ec46

[platforms] improve rocm debugging info (#14257)

youkaichao committed 1 year ago

Verified 6eaf9302

[V1] EP/TP MoE + DP Attention (#13931)

tlrmchlsmth committed 1 year ago

Verified 72c62eae

[Model] New model support for Phi-4-multimodal-instruct (#14119)

congcongchen123 committed 1 year ago

Verified 0a995d54

[V1][Bugfix] Do not reset prefix caching metrics (#14235)

comaniac committed 1 year ago

Verified ade3f7d9

[Bugfix] Fix gptq_marlin for deepseek-v3 (#13750)

rainkert committed 1 year ago

Verified 0df25101

Disable GPTQ AllSpark kernels for CUDA Compiler < 12.0 (#14157)

mgoin committed 1 year ago

Verified e123aafd

Moved numba from common requirements to cuda/rocm specific requirements (#14199)

npanpaliya committed 1 year ago

Verified 5b143d33

[misc] announce china meetup (#14248)

youkaichao committed 1 year ago

Verified eb59b5a6

[V1][TPU] TPU multimodal model support for ragged attention (#14158)

mgoin committed 1 year ago

Verified fbfc3ee3

[ROCm] Disable a few more kernel tests that are broken on ROCm (#14145)

SageMoore committed 1 year ago

Verified 3e1d2236

Clean up unused padding_idx variables across many model definitions (#13240)

tlrmchlsmth committed 1 year ago

Verified 4f5b059f

[Security] Serialize using safetensors instead of pickle in Mooncake Pipe (#14228)

KuntaiDu committed 1 year ago

Verified 288ca110

[v1][Metrics] Add design doc (#12745)

markmc committed 1 year ago

Verified c2bd2196

[Docs] Update Dockerfile dependency image (#14215)

mgoin committed 1 year ago

Verified 550c7ba3

[Frontend] Do `prompt_logprobs` clamping for chat as well as completions (#14225)

hmellor committed 1 year ago

Verified e5b2f160

Fix performance when `--generation-config` is not `None` (#14223)

hmellor committed 1 year ago

Verified 9badee53

[TPU][Profiler] Support start_profile/stop_profile in TPU worker (#13988)

lsy323 committed 1 year ago

Verified beebf474

add cutlass support for blackwell fp8 gemm (#13798)

kushanam committed 1 year ago

Verified f89978ad

[V1][Molmo] Fix get_multimodal_embeddings() in molmo.py (#14161)

lk-chen committed 1 year ago

Verified b3cf368d

[V0][Metrics] Deprecate some questionable request time metrics (#14135)

markmc committed 1 year ago

Verified c8525f06

[V1][BugFix] Fix remaining sync engine client shutdown errors/hangs (#13869)

njhill committed 1 year ago

Verified 5db6b2c9

Older