Improve openai compatible api for langchain support #1246
merrymercy
merged
24 commits into
lm-sys:main
from
andy-yang-1:langchain-support
langchain support
caf130ef
support batch embedding & completion
290d5c13
modify protocal
f5e060d0
batch token num
933c385c
add langchain use cases
401e058d
fix tiktoken error
2f8eb3aa
modify time out
5e15c882
explanation for examples
aaf527f1
andy-yang-1
changed the title 
enhance batch embedding
f1c55786
modify batch size to prevent oom
c9e6017e
fix eos token problem
ebe0b229
reduce memory pressure
61f854f1
remove debug print
049a6c2a
support mpt
bcbb92c9
add support for adapter
c3d6d4ea
formatting
4df8d878
merrymercy
commented
on 2023-05-19
merge & change time out
2033fc14
add dependency & timeout constant & fix style
014e122e
fix pad token
7dc69e0e
normalize for embedding
6276fe82
fix typo
8b181799
change is_vicuna to is_llama & mv to worker
7d91f10b
support t5
25ecc535
add unittest for langchain support
fae4087b
merrymercy
approved these changes
on 2023-05-20
Login to write a write a comment.
Login via GitHub