vllm
Add `/v1/chat/completions/batch` endpoint for batched chat completions
#38011
Merged
Overview
Commits
24
Changes
View On GitHub
Commits
  • Add batch requests to vllm api
    MatejRojec committed 11 days ago
  • Fix n>1 validation and echo condition for batched chat completions
    MatejRojec committed 11 days ago
  • Fix pre-commit warnings
    MatejRojec committed 11 days ago
  • Update system prompt in .sh file
    MatejRojec committed 11 days ago
  • Add edge case to manual tests
    MatejRojec committed 11 days ago
  • Improve test by making them more robust
    MatejRojec committed 11 days ago
  • Improve documentation and add ordering so results are alwaus ordered by index
    MatejRojec committed 11 days ago
  • Fix pre commit checks with ruff
    MatejRojec committed 11 days ago
  • Implement batch completions as a separate endpoint
    MatejRojec committed 11 days ago
  • Fix None request_id validation in to_chat_completion_request
    MatejRojec committed 11 days ago
  • Replace shell script example with Python script
    MatejRojec committed 11 days ago
  • Move render_batch_chat_request into OpenAIServingChat
    MatejRojec committed 11 days ago
  • Move render_batch_chat_request after render_chat_request
    MatejRojec committed 11 days ago
  • Move batch chat completion logic into OpenAIServingChatBatch subclass
    MatejRojec committed 11 days ago
  • Move import json to top of test_chat_completion.py
    MatejRojec committed 11 days ago
  • Move batch chat completion tests to dedicated test file
    MatejRojec committed 11 days ago
  • Add separate state for batch chat completion handler
    MatejRojec committed 11 days ago
  • Fix mypy errors in OpenAIServingChatBatch
    MatejRojec committed 11 days ago
  • Add BatchChatCompletionRequest to ChatLikeRequest type alias
    MatejRojec committed 11 days ago
  • Add validator to BatchChatCompletionRequest
    MatejRojec committed 11 days ago
  • Fix schemathesis failures for batch chat completion endpoint
    MatejRojec committed 11 days ago
  • Merge branch 'main' into feature/add-batch-requests-to-chat-completions-api
    MatejRojec committed 11 days ago
  • Fix ProcessorInputs import path in batch_serving.py
    MatejRojec committed 11 days ago
  • Fix EngineInput import in batch_serving.py
    MatejRojec committed 11 days ago
FAQ Terms Privacy Refunds Impressum
Loading