llama.cpp
ci: bench: support sse and fix prompt processing time / server: add tokens usage in stream OAI response
#6495
Merged
Merged
ci: bench: support sse and fix prompt processing time / server: add tokens usage in stream OAI response #6495
phymbert
merged
6 commits into
master
from
hp/server/bench/sse
ci: bench: support sse and fix prompt processing time
713fa986
ci: bench: README.md EOL
1534d903
ci: bench: remove total pp and tg as it is not accurate
36940266
ci: bench: fix case when there is no token generated
59dc4bbb
ci: bench: change to the 95 percentile for pp and tg as it is closer …
b6b50b11
ci: bench: fix finish reason rate
8789e17f
ngxson
approved these changes
on 2024-04-04
Login to write a write a comment.
Login via GitHub