vllm-project/vllm

Pull Requests Commits

Merge branch 'main' into optimize-prefix-caching-scheduling

zhuohan123 committed 2 years ago

d5bf492f

[FRONTEND] OpenAI `tools` support named functions (#5032)

br3no committed 2 years ago

Verified f775a07e

New CI template on AWS stack (#5110)

khluu committed 2 years ago

Verified 4f0d17c0

[Misc]: Implement CPU/GPU swapping in BlockManagerV2 (#3834)

Kaiyang-Chen committed 2 years ago

Verified 10c38e3e

[CI/BUILD] enable intel queue for longer CPU tests (#4113)

zhouyuan committed 2 years ago

Verified cafb8e06

[Kernel] Pass a device pointer into the quantize kernel for the scales (#5159)

tlrmchlsmth committed 2 years ago

Verified cbb2f59c

[Core] Remove unnecessary copies in flash attn backend (#5138)

Yard1 committed 2 years ago

Verified 0ab278ca

[Core] Support image processor (#4197)

DarkLight1337 committed 2 years ago

Verified 7a64d24a

zhuohan123 committed 2 years ago

8c7bab79

[Misc] Simplify code and fix type annotations in `conftest.py` (#5118)

DarkLight1337 committed 2 years ago

Verified dfbe60dc

[Kernel][ROCm][AMD] enable fused topk_softmax kernel for moe layer (#4927)

divakar-amd committed 2 years ago

Verified a66cf40b

[Frontend][OpenAI] Support for returning max_model_len on /v1/models response (#4643)

Avinash-Raj committed 2 years ago

Verified f790ad3c

Update test_ignore_eos (#4898)

simon-mo committed 2 years ago

Verified ed59a7ed

zhuohan123 committed 2 years ago

1936d7ba

Fix hashing logic for non-full blocks

zhuohan123 committed 2 years ago

996cf2de

[BugFix] Prevent `LLM.encode` for non-generation Models (#5184)

robertgshaw2-redhat committed 2 years ago

Verified 044793d8

[Bugfix]: Fix issues related to prefix caching example (#5177) (#5180)

Delviet committed 2 years ago

Verified c2d6d2f9

[Bugfix] Remove deprecated @abstractproperty (#5174)

zhuohan123 committed 2 years ago

Verified 8279078e

[Feature][Kernel] Support bitsandbytes quantization and QLoRA (#4776)

chenqianfzh committed 2 years ago

Verified b9c0605a

[Bugfix] Fix call to init_logger in openai server (#4765)

NadavShmayo committed 2 years ago

Verified 37464a0f

[Minor] Fix the path typo in loader.py: save_sharded_states.py -> save_sharded_state.py (#5151)

dashanji committed 2 years ago

Verified c3540728

[Kernel] Update Cutlass fp8 configs (#5144)

varun-sundar-rabindranath committed 2 years ago

Verified f081c3ce

[Kernel] Refactor CUTLASS kernels to always take scales that reside on the GPU (#5137)

tlrmchlsmth committed 2 years ago

Verified 260d119e

[CI/Build] CMakeLists: build all extensions' cmake targets at the same time (#5034)

dtrifiro committed 2 years ago

Verified a360ff80

[Build] Guard against older CUDA versions when building CUTLASS 3.x kernels (#5168)

tlrmchlsmth committed 2 years ago

Verified 1197e021

[Doc] Add checkmark for GPTBigCodeForCausalLM LoRA support (#5171)

njhill committed 2 years ago

Verified 65757911

[Model] Enable FP8 QKV in MoE and refine kernel tuning script (#5039)

comaniac committed 2 years ago

Verified e9899fb7

[Misc]: optimize eager mode host time (#4196)

FuncSherl committed 2 years ago

Verified a377f0bd

Revert "[Kernel] Marlin_24: Ensure the mma.sp instruction is using the ::ordered_metadata modifier (introduced with PTX 8.5)" (#5149)

simon-mo committed 2 years ago

Verified e9d3aa04

[Model] Support MAP-NEO model (#5081)

xingweiqu committed 2 years ago

Verified a22dea54

Older