vllm
03518f90 - [Core] Extensible KV cache: treat a connector's custom memory pool as unsupported

Commit
6 days ago
[Core] Extensible KV cache: treat a connector's custom memory pool as unsupported Upstream #49300 lets a KV connector (Mooncake Store with `custom_mem_pool`, NVLINK or BAREX) supply the pool the KV cache is allocated from, so the memory it registers comes from its own allocator. The extensible KV cache is mapped through the driver and never allocates from a torch pool, so the request would be ignored silently. Resolve it like `kv_cache_memory_bytes`: the feature is off by default when a connector (directly or under MultiConnector) sets `custom_mem_pool`, and an explicit request is an error. Mooncake already rejects the pool together with sleep mode and the cumem allocator for the same reason. Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Author
Committer
Parents
Loading