vllm-project/vllm

Pull Requests Commits

youkaichao committed 2 years ago

617fb893

[Bug Fix] Illegal memory access, FP8 Llama 3.1 405b (#6852)

LucasWilkinson committed 2 years ago

Verified 55712941

[Frontend] Factor out code for running uvicorn (#6828)

DarkLight1337 committed 2 years ago

Verified 981b0d56

[TPU] Support collective communications in XLA devices (#6813)

WoosukKwon committed 2 years ago

Verified d09b94ca

enforce eager mode with bnb quantization temporarily (#6846)

chenqianfzh committed 2 years ago

Verified bb549467

Update README.md (#6847)

gurpreet-dhami committed 2 years ago

Verified b5f49ee5

[Doc] Update SkyPilot doc for wrong indents and instructions for update service (#4283)

Michaelvll committed 2 years ago

Verified 150a1ffb

[Doc] Add Nemotron to supported model docs (#6843)

mgoin committed 2 years ago

Verified 281977bd

[Hardware] [Intel] Enable Multiprocessing and tensor parallel in CPU backend and update documentation (#6125)

bigPYJ1151 committed 2 years ago

Verified 3bbb4936

[Misc][TPU] Support TPU in initialize_ray_cluster (#6812)

WoosukKwon committed 2 years ago

Verified aa486779

[Build/CI][ROCm] Minor simplification to Dockerfile.rocm (#6811)

WoosukKwon committed 2 years ago

Verified 71734f1b

[Bugfix][Kernel] Promote another index to int64_t (#6838)

tlrmchlsmth committed 2 years ago

Verified 50704f52

[Model] Support Nemotron models (Nemotron-3, Nemotron-4, Minitron) (#6611)

mgoin committed 2 years ago

Verified 07278c37

[doc][debugging] add known issues for hangs (#6816)

youkaichao committed 2 years ago

Verified 85ad7e2d

[Core] Use array to speedup padding (#6779)

peng1999 committed 2 years ago

Verified 89a84b0b

[Bugfix] [Easy] Fixed a bug in the multiprocessing GPU executor. (#6770)

eaplatanios committed 2 years ago

Verified 084a01fd

Fix ReplicatedLinear weight loading (#6793)

qingquansong committed 2 years ago

Verified 062a1d0f

[ci] Mark tensorizer as soft fail and separate from grouped test (#6810)

khluu committed 2 years ago

Verified 2eb9f4ff

[ci][distributed] fix flaky tests (#6806)

youkaichao committed 2 years ago

Verified 443c7cf4

[Core] Fix ray forward_dag error mssg (#6792)

rkooo567 committed 2 years ago

Verified 1adddb14

[Docs] Publish 5th meetup slides (#6799)

WoosukKwon committed 2 years ago

Verified b7215de2

[doc][distributed] improve multinode serving doc (#6804)

youkaichao committed 2 years ago

Verified f3ff63c3

[Bugfix] Fix empty (nullptr) channelwise scales when loading wNa16 using compressed tensors (#6798)

LucasWilkinson committed 2 years ago

Verified cd7edc4e

[Doc] Add documentations for nightly benchmarks (#6412)

KuntaiDu committed 2 years ago

Verified 6a1e25b1

[Bugfix] Add synchronize to prevent possible data race (#6788)

tlrmchlsmth committed 2 years ago

Verified 95db75de

[Bugfix] Fix `kv_cache_dtype=fp8` without scales for FP8 checkpoints (#6761)

mgoin committed 2 years ago

Verified 65b1f121

[ Misc ] `fp8-marlin` channelwise via `compressed-tensors` (#6524)

robertgshaw2-redhat committed 2 years ago

Verified 889da130

[Bugfix] Add image placeholder for OpenAI Compatible Server of MiniCPM-V (#6787)

HwwwwwwwH committed 2 years ago

Verified b75e314f

[Bugfix] Fix encoding_format in examples/openai_embedding_client.py (#6755)

CatherineSue committed 2 years ago

Verified 316a41ac

[Bugfix] Fix awq_marlin and gptq_marlin flags (#6745)

alexm-redhat committed 2 years ago

Verified 0310029a

Older