vllm-project/vllm

Pull Requests Commits

Merge remote-tracking branch 'origin/main' into fp8_ep_dp

tlrmchlsmth committed 1 year ago

1236aebf

[Bugfix][Model] Attempt to fix eagle in V0. (#18978)

gshtras committed 1 year ago

Verified ca2f6b9c

[Frontend] enable custom logging for the uvicorn server (OpenAI API server) (#18403)

fpaupier committed 1 year ago

Verified 20133cfe

[Model] enable data parallel for Llama4 vision encoder (#18368)

jennyyyyzhen committed 1 year ago

Verified ebb1ec93

[doc] add pytest tips (#19010)

reidliu41 committed 1 year ago

Verified 5b168b6d

[Core] Support inplace model weights loading (#18745)

22quinn committed 1 year ago

Verified 9760fd8f

[Bugfix][Nixl] Fix DP Metadata Handshake (#19008)

robertgshaw2-redhat committed 1 year ago

Verified b9f61e13

[Misc] reuse num_tokens_across_dp of get_dp_padding to avoid unnecessary dp all reduce in set_forward_context (#18935)

izhuhaoran committed 1 year ago

Verified d6fd3a33

[doc] wrong output (#19000)

reidliu41 committed 1 year ago

Verified 432ec992

[BugFix] Fix incorrect metrics shutdown error log message (#18992)

njhill committed 1 year ago

Verified 2b102d51

[BugFix] fix data parallel construct ipv6 url addres (#18991)

lengrongfu committed 1 year ago

Verified aa54a7bf

Let max_num_batched_tokens use human_readable_int for large numbers (#18968)

mgoin committed 1 year ago

Verified 2ad6194a

[doc] small fix - mkdocs (#18996)

reidliu41 committed 1 year ago

Verified c594cbf5

[LoRA] Support dynamically initialize `packed_modules_mapping` for VLM with arbitrary components (#18987)

Isotr0py committed 1 year ago

Verified a35ca765

[Core] Rework dtype resolution (#18751)

DarkLight1337 committed 1 year ago

Verified 6aa8f9a4

[Bugfix] Fix EAGLE3 broken logits (#18909)

benchislett committed 1 year ago

Verified 1bc86a3d

[Misc][Benchmark] Add support for CustomDataset (#18511)

ekagra-ranjan committed 1 year ago

Verified bbfa0c61

[Misc] add return token strs for tokenize (#18941)

reidliu41 committed 1 year ago

Verified 20079c6e

[BugFix] Fix multi-node offline data-parallel (#18981)

njhill committed 1 year ago

Verified 9a1b9b99

[P/D] NixlConnector use cache device index for memory registration (#18969)

ptarasiewiczNV committed 1 year ago

Verified 8bf507d7

[ROCm][Kernel] Add gfx950 support for skinny gemms (#18010)

charlifu committed 1 year ago

Verified 306d6040

[Bugfix] Fix for issue 17396 (#18773)

frreiss committed 1 year ago

Verified f2c3f66d

[FEAT][ROCm] Add AITER grouped topk for DeepSeekV2 (#18825)

vllmellm committed 1 year ago

Verified 0f5e0d56

[BugFix] Pydantic part 2 (#18911)

ProExpertProg committed 1 year ago

Verified c55d8046

[doc] fix the list rendering issue - security.md (#18982)

reidliu41 committed 1 year ago

Verified 749f5bdd

[Neuron] Add Multi-Modal model support for Neuron (#18921)

aws-satyajith committed 1 year ago

Verified 2a50ef57

fix security issue of logging llm output (#18980)

luccafong committed 1 year ago

Verified b8b90479

[Bugfix]: Fix the incompatibility issue with Structured Outputs when Thinking is disabled (#18879)

chaunceyjiang committed 1 year ago

Verified ba5111f2

[Misc] Fix estimated max model len msg (#18966)

sarckk committed 1 year ago

Verified 1e123529

[Frontend] Add rerank support to run_batch endpoint (#16278)

pooyadavoodi committed 1 year ago

Verified dff80b0e

Older