vllm-project/vllm

Pull Requests Commits

add aggregator interface and abstract common logic

luccafong committed 1 year ago

98d535eb

[Misc][DP] support customized global logger for dp

luccafong committed 1 year ago

a46e2799

[fix]: remove data type hardcoding from gptoss model implementation (#23807)

nikhil-arm committed 1 year ago

Verified 064cac7b

[V0 Deprecation] Remove AsyncLLMEngine (#25025)

WoosukKwon committed 1 year ago

Verified e19bce40

[KV offload][1/N] Introduce an offloading component (#19848)

orozery committed 1 year ago

Verified 505805b6

[ROCm][AITER][Bugfix] Switch AITER to use PIECEWISE_AND_FULL compilation (#25104)

Rohan138 committed 1 year ago

Verified bbdc0f23

[ROCm][CI/Build] Use ROCm7.0 as the base (#25178)

gshtras committed 1 year ago

Verified dc340593

[spec decode] Fix MTP inference path for MiMo-7B model (#25136)

zixi-qi committed 1 year ago

Verified c4cb0af9

[Misc] Add codeowner for Transformers backend (#25180)

hmellor committed 1 year ago

Verified 1c3b1634

Enable Allgather/ReduceScatter backend for NaiveAllToAll (#23964)

wenscarl committed 1 year ago

Verified 2ea50e97

[Docs] Fix warnings in mkdocs build (continued) (#25163)

Zerohertz committed 1 year ago

Verified b419937c

[New Model] Support BertForTokenClassification / Named Entity Recognition (NER) task (#24872)

noooop committed 1 year ago

Verified 5f696c33

feat(api): Return 503 on /health when engine is dead (#24897)

dongbo910220 committed 1 year ago

Verified 67244c86

[PERF] Add `conv1d` metadata to GDN attn (#25105)

vadiklyutiy committed 1 year ago

Verified 072d7e53

[Kernel] Decouple Tile Size from Block Size in Triton Unified Attention Kernel (#21197)

jvlunteren committed 1 year ago

Verified 01a583fe

[Misc] Add kv-connector label (#25156)

NickLucche committed 1 year ago

Verified bc19d759

Refactor dense FP8 tensor/channel/block utils and add CT FP8 block (#21404)

mgoin committed 1 year ago

Verified fbd6523a

[Structured Output][Refactor] Move `apply_grammar_bitmask()` method from `ModelRunner` to structured output utils (#21999)

shen-shanshan committed 1 year ago

Verified 470484a4

[Misc] Clean up flags in `vllm bench serve` (#25138)

ywang96 committed 1 year ago

Verified 21da7334

[Bugfix][Mamba] - Fix Conv State Kernel FP32 Support (#24883)

Josephasafg committed 1 year ago

Verified 66072b36

Fix `validate-config` pre-commit check (#25157)

hmellor committed 1 year ago

Verified 3ed1ec4a

Fix forward reference warning in documentation (#25150)

hmellor committed 1 year ago

Verified 5a33ae9a

[Docs] add the parallel sampling usage in LLMEngine and AsyncLLM (#24222)

gigit0000 committed 1 year ago

Verified c9ff9e6f

[Docs] Fix pooling-params doc references in openai_compatible_server.md (#24939)

yankay committed 1 year ago

Verified eaffe448

Move `StructuredOutputsConfig` from `config/__init__.py` to `config/structured_outputs.py` (#25153)

hmellor committed 1 year ago

Verified 8ed039d5

[Model] Improve Pooling Model (#25149)

jeejeelee committed 1 year ago

Verified 37970105

[Frontend] Support setting logprobs to -1 (#25031)

chaunceyjiang committed 1 year ago

Verified cc935fdd

silu-v1: Fix EPS not being used during max-reduction (#25069)

elvircrn committed 1 year ago

Verified abdfcd4f

Fix: Add explicit #include <omp.h> for OpenMP compatibility on certain toolchains (#24951)

ihb2032 committed 1 year ago

Verified 4f02b77d

[Chore] Cleanup guided namespace, move to structured outputs config (#22772)

aarnphm committed 1 year ago

Verified 29283e89

Older