vllm-project/vllm

Pull Requests Commits

Support moe_wna16 as well

mgoin committed 1 year ago

243408b6

Merge branch 'main' into mla-support-awq-marlin

mgoin committed 1 year ago

b8510f10

[Bugfix] Allow fallback to AWQ from AWQMarlin at per-layer granularity (#13119)

mgoin committed 1 year ago

Verified 09972e71

[CORE] [QUANT] Support for GPTQModel's `dynamic` quantization per module override/control (#7086)

Qubitium committed 1 year ago

Verified 36a08630

[CI/Build] Use mypy matcher for pre-commit CI job (#13162)

russellb committed 1 year ago

Verified 2c2b560f

Introduce VLLM_CUDART_SO_PATH to allow users specify the .so path (#12998)

houseroad committed 1 year ago

Verified 042c3419

[Misc] Delete unused LoRA modules (#13151)

jeejeelee committed 1 year ago

Verified 82cabf53

[Frontend] Generate valid tool call IDs when using `tokenizer-mode=mistral` (#12332)

rafvasq committed 1 year ago

Verified 314cfade

[Bugfix] Fix num video tokens calculation for Qwen2-VL (#13148)

DarkLight1337 committed 1 year ago

Verified 985b4a2b

[Bug] [V1] Try fetching stop_reason from EngineOutput before checking the request (#13108)

bnellnm committed 1 year ago

Verified f4d97e4f

[Misc] AMD Build Improvements (#12923)

842974287 committed 1 year ago

Verified f1042e86

Further reduce the HTTP calls to huggingface.co (#13107)

maxdebayser committed 1 year ago

Verified 7c4033ac

Bump actions/setup-python from 5.3.0 to 5.4.0 (#12672)

dependabot[bot] committed 1 year ago

Verified d59def47

Bump helm/chart-testing-action from 2.6.1 to 2.7.0 (#12463)

dependabot[bot] committed 1 year ago

Verified 0c7d9eff

Bump actions/stale from 9.0.0 to 9.1.0 (#12462)

dependabot[bot] committed 1 year ago

Verified dd3b4a01

Bump helm/kind-action from 1.10.0 to 1.12.0 (#11612)

dependabot[bot] committed 1 year ago

Verified a0597c6b

[Neuron][Kernel] Support Longer Sequences in NKI-based Flash PagedAttention and Improve Efficiency (#12921)

lingfanyu committed 1 year ago

Verified e92694b6

[ci] Add more source file dependencies for some tests (#13123)

khluu committed 1 year ago

Verified 842b0fd4

[Model] IBM/NASA Prithvi Geospatial model (#12830)

christian-pinto committed 1 year ago

Verified 974dfd49

[RFC][vllm-API] Support tokenizer registry for customized tokenizer in vLLM (#12518)

youngkent committed 1 year ago

Verified 3ee696a6

[Misc] Move pre-commit suggestion back to the end (#13114)

russellb committed 1 year ago

Verified 72c2b68d

Combine loader _process_weights_after_loading

mgoin committed 1 year ago

09318cae

[Bugfix] Guided decoding falls back to outlines when fails to import xgrammar (#12976)

terrytangyuan committed 1 year ago

Verified 14ecab5b

[Doc] Improve OpenVINO installation doc (#13102)

hmellor committed 1 year ago

Verified deb6c1c6

[CI/Build][Bugfix] Fix CPU backend default threads num (#13077)

bigPYJ1151 committed 1 year ago

Verified 565c1efa

Fix initializing GGUF weights for ColumnParallelLinear when using tensor parallel > 1 (#13023)

SzymonOzog committed 1 year ago

Verified 2b25b7d2

[BugFix] Pop instead of del CUDA_VISIBLE_DEVICES (#12962)

HollowMan6 committed 1 year ago

Verified 6c4dbe23

[Bugfix] Do not use resource module on Windows (#12858) (#13029)

MoonRide303 committed 1 year ago

Verified 21f5d50f

[Misc] Fix typo at comments at metrics.py (#13024)

je1lee committed 1 year ago

Verified bf3e0521

Set `torch_dtype` in `TransformersModel` (#13088)

hmellor committed 1 year ago

Verified ad977635

Older