vllm
[Core] Reuse empty block lists whenever possible in KVCacheBlocks to mitigate GC costs
#24964
Merged
Merged
[Core] Reuse empty block lists whenever possible in KVCacheBlocks to mitigate GC costs #24964
njhill
merged
12 commits into
vllm-project:main
from
Jialin:gc_none_block
Jialin
changed the title
Jialin
changed the title
list -> tuple
049b5a5a
change more interfaces
6f905c3d
code comment
ba941262
Address comments
371f2f79
tuple[list[KVCacheBlock]] -> tuple[Sequence[KVCacheBlock]]
dbfbc6c6
Fix import
e08e898a
format
e1bba140
Aoivd unnecessary iterations and copies in KVCacheBlocks.__add__
692acf69
Get rid of custom type and use Sequence[KVCacheBlock] directly
4b781afb
fix precommit error
80d758bc
houseroad
commented
on 2025-10-14
houseroad
approved these changes
on 2025-10-14
Address comments
bad5bf3e
Remove unrelated changes
1f110b14
njhill
approved these changes
on 2025-10-14
Reviewers
Assignees
No one assigned
Labels
frontend
ready
v1
multi-modality
Milestone
No milestone
Login to write a write a comment.
Login via GitHub