vllm-project/vllm

Pull Requests Commits

zhuohan123 committed 2 years ago

1936d7ba

Fix hashing logic for non-full blocks

zhuohan123 committed 2 years ago

996cf2de

[Kernel] Refactor CUTLASS kernels to always take scales that reside on the GPU (#5137)

tlrmchlsmth committed 2 years ago

Verified 260d119e

[CI/Build] CMakeLists: build all extensions' cmake targets at the same time (#5034)

dtrifiro committed 2 years ago

Verified a360ff80

[Build] Guard against older CUDA versions when building CUTLASS 3.x kernels (#5168)

tlrmchlsmth committed 2 years ago

Verified 1197e021

[Doc] Add checkmark for GPTBigCodeForCausalLM LoRA support (#5171)

njhill committed 2 years ago

Verified 65757911

[Model] Enable FP8 QKV in MoE and refine kernel tuning script (#5039)

comaniac committed 2 years ago

Verified e9899fb7

[Misc]: optimize eager mode host time (#4196)

FuncSherl committed 2 years ago

Verified a377f0bd

Revert "[Kernel] Marlin_24: Ensure the mma.sp instruction is using the ::ordered_metadata modifier (introduced with PTX 8.5)" (#5149)

simon-mo committed 2 years ago

Verified e9d3aa04

[Model] Support MAP-NEO model (#5081)

xingweiqu committed 2 years ago

Verified a22dea54

Fix cutlass sm_90a vesrion in CMakeList

simon-mo committed 2 years ago

533c2177

[Kernel] Marlin_24: Ensure the mma.sp instruction is using the ::ordered_metadata modifier (introduced with PTX 8.5) (#5136)

alexm-redhat committed 2 years ago

Verified 6d21fa1c

[Bugfix] Avoid Warnings in SparseML Activation Quantization (#5120)

robertgshaw2-redhat committed 2 years ago

Verified b35be540

[Build] Disable sm_90a in cu11 (#5141)

simon-mo committed 2 years ago

Verified 45a1a69b

Bump version to v0.4.3 (#5046)

simon-mo committed 2 years ago

Verified 87a658c8

add doc about serving option on dstack (#3074)

deep-diver committed 2 years ago

Verified 429d8972

[Doc] Use intersphinx and update entrypoints docs (#5125)

DarkLight1337 committed 2 years ago

Verified a9bcc7af

[Misc] remove duplicate definition of `seq_lens_tensor` in model_runner.py (#5129)

ita9naiwa committed 2 years ago

Verified d79d9eaa

[CI/Build] increase wheel size limit to 200 MB (#5130)

youkaichao committed 2 years ago

Verified f758505c

[Bugfix] Automatically Detect SparseML models (#5119)

robertgshaw2-redhat committed 2 years ago

Verified d910816c

[BUGFIX] [FRONTEND] Correct chat logprobs (#5029)

br3no committed 2 years ago

Verified 87d41c84

[CI/Build] Docker cleanup functionality for amd servers (#5112)

okakarpa committed 2 years ago

Verified e07aff9e

[Bugfix] gptq_marlin: Ensure g_idx_sort_indices is not a Parameter (#5108)

alexm-redhat committed 2 years ago

Verified 5bf185a1

[Doc][Build] update after removing vllm-nccl (#5103)

youkaichao committed 2 years ago

Verified 4fbcb0f2

[Bugfix] logprobs is not compatible with the OpenAI spec #4795 (#5031)

Etelis committed 2 years ago

Verified 7c3604fb

[Core] Avoid the need to pass `None` values to `Sequence.inputs` (#5099)

DarkLight1337 committed 2 years ago

Verified b1c25563

[Bugfix][CI/Build] Fix codespell failing to skip files in `git diff` (#5097)

DarkLight1337 committed 2 years ago

Verified eb6c50cd

[Bugfix][CI/Build] Fix test and improve code for `merge_async_iterators` (#5096)

DarkLight1337 committed 2 years ago

Verified eecd8643

[Doc]Replace deprecated flag in readme (#4526)

ronensc committed 2 years ago

Verified ae495c74

[Core] Cross-attention KV caching and memory-management (towards eventual encoder/decoder model support) (#4837)

afeldman-nm committed 2 years ago

Verified 4238bc82

Older