vllm
[MISC] Add prefix cache hit rate to metrics
#7606
Merged

Comment changes are shownComment changes are hidden
  • tests
    • core/block
      • File
        test_prefix_caching_block.py
    • prefix_caching
      • File
        test_prefix_caching.py
  • vllm
    • core
      • block
        • File
          common.py
        • File
          cpu_gpu_block_allocator.py
        • File
          interfaces.py
        • File
          naive_block.py
        • File
          prefix_caching_block.py
      • File
        block_manager_v1.py
      • File
        block_manager_v2.py
      • File
        embedding_model_block_manager.py
      • File
        evictor_v2.py
      • File
        interfaces.py
      • File
        scheduler.py
    • engine
      • File
        llm_engine.py
      • File
        metrics.py
      • File
        metrics_types.py

Loading comments...