PR #24010 server: add KV cache metrics

server: add KV cache metrics #24010

lvsijian8 wants to merge 1 commit into ggml-org:master from lvsijian8:fix/kv-cache-metrics

server: add KV cache metrics

ab354c6d

lvsijian8 requested a review 19 days ago

lvsijian8 requested a review from

ggerganov 19 days ago

github-actions added examples

github-actions added python

github-actions added server

Reviewers

ggerganov

Assignees

No one assigned

Labels

examples python server

Milestone

No milestone