vllm-project/vllm

Pull Requests Commits

Merge branch 'main' into woosuk-jf

WoosukKwon committed 1 year ago

bcf3c823

[Bugfix] Prioritize dtype in root config before checking text config (#17629)

DarkLight1337 committed 1 year ago

Verified 2858830c

Add full API docs and improve the UX of navigating them (#17485)

hmellor committed 1 year ago

Verified d6484ef3

[Misc] V0 fallback for `--enable-prompt-embeds` (#17615)

DarkLight1337 committed 1 year ago

Verified 46fae69c

Merge branch 'main' into woosuk-jf

WoosukKwon committed 1 year ago

a01af39a

[Bugfix] Fix broken Qwen2.5-omni tests (#17613)

Isotr0py committed 1 year ago

Verified f66f1e0f

[Core] Gate `prompt_embeds` behind a feature flag (#17607)

DarkLight1337 committed 1 year ago

Verified 887d7af8

[Bugfix][ROCm] Using device_type because on ROCm the API is still torch.cuda (#17601)

gshtras committed 1 year ago

Verified a9284245

[Build/CI] Upgrade CUTLASS to 3.9.1 (#17602)

tlrmchlsmth committed 1 year ago

Verified c8386fa6

[Frontend][TPU] Add TPU default max-num-batched-tokens based on device name (#17508)

Chenyaaang committed 1 year ago

Verified 87baebeb

[Quantizaton] [AMD] Add support for running DeepSeek int8 w8a8 MoE on ROCm (#17558)

rasmith committed 1 year ago

Verified e3d0a1d1

Update test requirements to CUDA 12.8 (#17576)

22quinn committed 1 year ago

Verified d47b605e

[Neuron][Build] Require setuptools >= 77.0.3 for PEP 639 (#17603)

liangfu committed 1 year ago

Verified 22c6f639

[release] Add command to clean up Docker containers/images in TPU release machine (#17606)

khluu committed 1 year ago

Verified 3ec97e2c

fix typo in logging (#17605)

ehartford committed 1 year ago

Verified 9b103a1d

[easy] Print number of needed GPUs in skip message (#17594)

zou3519 committed 1 year ago

Verified b90b0852

[Hardware][AMD] Improve OAM device ID + llama4 Maverick MOE tuning (#16263)

xw285cornell committed 1 year ago

Verified 9352cdb5

Add NVIDIA TensorRT Model Optimizer in vLLM documentation (#17561)

Edwardf0t1 committed 1 year ago

Verified 182f40ea

permute/unpermute kernel for moe optimization (#14568)

CalebDu committed 1 year ago

Verified 3e887d2e

[BugFix][Attention] Fix sliding window attention in V1 giving incorrect results (#17574)

LucasWilkinson committed 1 year ago

Verified 0f87d8f7

[Bugfix] fix tmp_out and exp_sums dimensions (#17438)

hliuca committed 1 year ago

Verified 4c33d673

[Misc] Clean up input processing (#17582)

DarkLight1337 committed 1 year ago

Verified cb234955

[doc] miss result (#17589)

reidliu41 committed 1 year ago

Verified 3a500cd0

Support W8A8 INT8 MoE for compressed-tensors (#16745)

mgoin committed 1 year ago

Verified 868c546d

[Security] Fix image hash collision (#17378)

DarkLight1337 committed 1 year ago

Verified 99404f53

Automatically tell users that dict args must be valid JSON in CLI (#17577)

hmellor committed 1 year ago

Verified 785d75a0

[doc] add the print result (#17584)

reidliu41 committed 1 year ago

Verified 6d1479ca

add more pytorch related tests for torch nightly (#17422)

yangw-dev committed 1 year ago

Verified b8b0859b

[Misc] Rename assets for testing (#17575)

DarkLight1337 committed 1 year ago

Verified d7543862

[BugFix] Fix Memory Leak (#17567)

robertgshaw2-redhat committed 1 year ago

Verified c777df79

Older