llama.cpp
server: benchmark: chat/completions scenario and other llm servers comparison
#5941
Merged
Merged
server: benchmark: chat/completions scenario and other llm servers comparison #5941
phymbert
merged
15 commits into
master
from
hp/server/bench/init
server: bench: Init a bench scenario with K6
68d1d8fe
phymbert
changed the title
server: bench: EOL EOF
0b822b6a
phymbert
changed the title 
server: bench: PR feedback and improved k6 script configuration
548bc963
server: bench: remove llamacpp_completions_tokens_seconds as it inclu…
ab0a59d6
server: bench: fix doc
f425240e
server: bench: change gauge custom metrics to trend
bed1cdda
server: bench: change gauge custom metrics to trend
572758a6
server: bench: doc add an option to debug http request
06e225f8
server: bench: filter dataset too short and too long sequences
a4b0d107
server: bench: allow to filter out conversation in the dataset based …
29c635b4
server: bench: fix assistant message sent instead of user message
ba7114c0
server: bench: fix assistant message sent instead of user message
c4d1b5aa
Merge branch 'master' into hp/server/bench/init
5d25f748
server : add defrag thold parameter
52c76d57
ggerganov
approved these changes
on 2024-03-09
ngxson
approved these changes
on 2024-03-09
server: bench: select prompts based on the current iteration id not r…
6bfb80eb
Login to write a write a comment.
Login via GitHub