vllm
Add `/v1/chat/completions/batch` endpoint for batched chat completions
#38011
Merged
Merged
Add `/v1/chat/completions/batch` endpoint for batched chat completions #38011
DarkLight1337
merged
24 commits into
vllm-project:main
from
MatejRojec:feature/add-batch-requests-to-chat-completions-api
MatejRojec
changed the title 
MatejRojec
changed the title
DarkLight1337
commented
on 2026-03-25
DarkLight1337
commented
on 2026-03-25
DarkLight1337
commented
on 2026-03-25
DarkLight1337
commented
on 2026-03-25
DarkLight1337
commented
on 2026-03-25
DarkLight1337
approved these changes
on 2026-03-25
disabled auto-merge 8 days ago
Head branch was pushed to by a user without write access
Head branch was pushed to by a user without write access
DarkLight1337
commented
on 2026-03-25
Add batch requests to vllm api
7d93bde1
Fix n>1 validation and echo condition for batched chat completions
47e124ef
Fix pre-commit warnings
b8a91ffc
Update system prompt in .sh file
eb2e5313
Add edge case to manual tests
2c76e2af
Improve test by making them more robust
6e268e9f
Improve documentation and add ordering so results are alwaus ordered …
724baee7
Fix pre commit checks with ruff
0d2848a5
Implement batch completions as a separate endpoint
09177f48
Fix None request_id validation in to_chat_completion_request
819052d8
Replace shell script example with Python script
cf107e3a
Move render_batch_chat_request into OpenAIServingChat
47b92b9d
Move render_batch_chat_request after render_chat_request
27706be1
Move batch chat completion logic into OpenAIServingChatBatch subclass
158ad59c
Move import json to top of test_chat_completion.py
d199a6f5
Move batch chat completion tests to dedicated test file
b92996a0
Add separate state for batch chat completion handler
92b996af
Fix mypy errors in OpenAIServingChatBatch
9bd033aa
Add BatchChatCompletionRequest to ChatLikeRequest type alias
f2600beb
Add validator to BatchChatCompletionRequest
bc23224a
DarkLight1337
approved these changes
on 2026-03-25
Fix schemathesis failures for batch chat completion endpoint
eddd17bd
disabled auto-merge 8 days ago
Head branch was pushed to by a user without write access
Head branch was pushed to by a user without write access
Merge branch 'main' into feature/add-batch-requests-to-chat-completio…
434a529a
Fix ProcessorInputs import path in batch_serving.py
2dc00654
Fix EngineInput import in batch_serving.py
cc69644a
DarkLight1337
approved these changes
on 2026-03-26
Reviewers
Assignees
No one assigned
Labels
documentation
frontend
ready
Milestone
No milestone
Login to write a write a comment.
Login via GitHub