llama.cpp
server: Add "tokens per second" information in the backend
#10548

Merged

server: Add "tokens per second" information in the backend #10548

ngxson merged 12 commits into ggml-org:master from lhpqaq:token

add cmake rvv support

4278480a

Merge branch 'ggerganov:master' into master

3f6406f9

Merge branch 'ggerganov:master' into master

daab1416

Merge branch 'ggerganov:master' into master

2c96bd24

github-actions added examples

github-actions added server

lhpqaq force pushed from 12558273 to 44f54740 297 days ago

lhpqaq changed the title ~~server: Add "tokens per second" information in the Web UI~~ server: Add "tokens per second" information in the backend 297 days ago

add timings

fb105215

lhpqaq force pushed from b4ff50b5 to fb105215 296 days ago

remove space

f766ae12

update readme

c98e9a71

fix

ce32516f

ngxson requested changes on 2024-11-30

Merge branch 'ggerganov:master' into token

c47c41cd

fix code

21f8b73d

remove empty line

1b301dbe

lhpqaq requested a review from

ngxson 295 days ago

add test

28d8c917

github-actions added python

ngxson approved these changes on 2024-12-02

ngxson merged 64ed2091 into master 293 days ago

lhpqaq deleted the token branch 291 days ago

Reviewers

ngxson

Assignees

No one assigned

Labels

examples python server

Milestone

No milestone

llama.cpp server: Add "tokens per second" information in the backend #10548 Merged

server: Add "tokens per second" information in the backend #10548

llama.cpp
server: Add "tokens per second" information in the backend
#10548

Merged