vllm
c386c43c - [V1][Metrics] Add per-request prompt/generation_tokens histograms (#12516)

Commit

1 year ago

[V1][Metrics] Add per-request prompt/generation_tokens histograms (#12516) Signed-off-by: Mark McLoughlin <markmc@redhat.com>

References

Author

markmc

Parents