[Core][Observability] Add KV cache residency metrics #27793
markmc
merged
8 commits into
vllm-project:main
from
shivampr:feat/kv-cache-idle-reuse-metrics
hmellor
commented
on 2025-10-30
markmc
commented
on 2025-11-26
markmc
approved these changes
on 2025-11-28
Add KV cache residency metrics with sampling
1df22c81
Update KV cache metrics flow to collect events in core and observe in…
17b9918d
Addresses review comments to simplify the implementation:
da773fa0
KV cache metrics cleanup: drop unused residency metric, simplify coll…
f6abbfe6
Addressed few review comments :
be715229
Fix test_kv_cache_metrics_surface_in_llm_get_metrics to trigger evict…
301f49bd
disabled auto-merge 207 days ago
Head branch was pushed to by a user without write access
Head branch was pushed to by a user without write access
Fix test_kv_cache_metrics_surface_in_llm_get_metrics to trigger evict…
ab1428bd
disabled auto-merge 207 days ago
Head branch was pushed to by a user without write access
Head branch was pushed to by a user without write access
Move KV cache metrics test from GPU-only engine integration to CPU-fr…
6a3f02ac
Reviewers
Assignees
No one assigned
Labels
documentation
ready
v1
Milestone
No milestone
Login to write a write a comment.
Login via GitHub