llama.cpp
server: Add "tokens per second" information in the backend
#10548

Merged

server: Add "tokens per second" information in the backend #10548

ngxson merged 12 commits into ggml-org:master from lhpqaq:token

add cmake rvv support

4278480a

Merge branch 'ggerganov:master' into master

3f6406f9

Merge branch 'ggerganov:master' into master

daab1416

Merge branch 'ggerganov:master' into master

2c96bd24

github-actions added examples

github-actions added server

lhpqaq force pushed 1 year ago

lhpqaq changed the title ~~server: Add "tokens per second" information in the Web UI~~ server: Add "tokens per second" information in the backend 1 year ago

add timings

fb105215

lhpqaq force pushed to fb105215 1 year ago

remove space

f766ae12

update readme

c98e9a71

fix

ce32516f

ngxson requested changes on 2024-11-30

Merge branch 'ggerganov:master' into token

c47c41cd

fix code

21f8b73d

remove empty line

1b301dbe

lhpqaq requested a review from

ngxson 1 year ago

add test

28d8c917

github-actions added python

ngxson approved these changes on 2024-12-02

ngxson merged 64ed2091 into master 1 year ago

lhpqaq deleted the token branch 1 year ago

Reviewers

ngxson

Assignees

No one assigned

Labels

examples python server

Milestone

No milestone

llama.cpp server: Add "tokens per second" information in the backend #10548 Merged

server: Add "tokens per second" information in the backend #10548

llama.cpp
server: Add "tokens per second" information in the backend
#10548

Merged