vllm
[ROCm][Engine] Fix GPU memory leaks in engine shutdown and test workaround for async KV prefix cache reset
#38503
Merged

[ROCm][Engine] Fix GPU memory leaks in engine shutdown and test workaround for async KV prefix cache reset #38503

AndreasKaratzas
AndreasKaratzas [Engine] Fix engine stalling when async KV offload defers block frees
e3cf764d
AndreasKaratzas AndreasKaratzas added rocm
AndreasKaratzas AndreasKaratzas added ready
mergify mergify added v1
gemini-code-assist
AndreasKaratzas
AndreasKaratzas
AndreasKaratzas AndreasKaratzas marked this pull request as ready for review 235 days ago
AndreasKaratzas AndreasKaratzas requested a review from WoosukKwon WoosukKwon 235 days ago
AndreasKaratzas AndreasKaratzas requested a review from robertgshaw2-redhat robertgshaw2-redhat 235 days ago
AndreasKaratzas AndreasKaratzas requested a review from njhill njhill 235 days ago
AndreasKaratzas AndreasKaratzas requested a review from ywang96 ywang96 235 days ago
AndreasKaratzas AndreasKaratzas requested a review from alexm-redhat alexm-redhat 235 days ago
AndreasKaratzas AndreasKaratzas requested a review from heheda12345 heheda12345 235 days ago
AndreasKaratzas AndreasKaratzas requested a review from ApostaC ApostaC 235 days ago
AndreasKaratzas AndreasKaratzas requested a review from orozery orozery 235 days ago
orozery
AndreasKaratzas Merge remote-tracking branch 'origin/main' into akaratza_fix_engine_a…
be4151ad
AndreasKaratzas Merge remote-tracking branch 'origin/main' into akaratza_fix_engine_a…
fb31eeb5
AndreasKaratzas [ROCm][CI] Adding explicitely fastsafetensors dep
8751ced7
AndreasKaratzas Merge remote-tracking branch 'origin/main' into akaratza_fix_engine_a…
54975d7b
AndreasKaratzas [ROCm] Fix GPU memory leaks in engine shutdown for in-process mode
bbef05b9
AndreasKaratzas [Core] Add synchronous flush of pending KV stores before prefix cache…
0b90a70a
AndreasKaratzas [ROCm][CI] Bump ROCm UCX branch to fix RIXL memory leak and create ne…
38dc6bfc
AndreasKaratzas AndreasKaratzas requested a review from gshtras gshtras 230 days ago
AndreasKaratzas AndreasKaratzas requested a review from tjtanaa tjtanaa 230 days ago
AndreasKaratzas AndreasKaratzas requested a review from NickLucche NickLucche 230 days ago
mergify mergify added ci/build
mergify mergify added kv-connector
AndreasKaratzas AndreasKaratzas changed the title [Engine] Fix engine stalling when async KV offload defers block frees [Engine] Fix GPU memory leaks in engine shutdown and add synchronous flush for prefix cache reset. 230 days ago
AndreasKaratzas AndreasKaratzas changed the title [Engine] Fix GPU memory leaks in engine shutdown and add synchronous flush for prefix cache reset. [Engine] Fix GPU memory leaks in engine shutdown and add synchronous flush for prefix cache reset 230 days ago
AndreasKaratzas
AndreasKaratzas [ROCm][RIXL] Updated version with mem leak fix
c2a6d267
AndreasKaratzas [CI] Fix OOM in torchrun MoE test by limiting max_model_len to 2048
c965356d
AndreasKaratzas [ROCm][RIXL] Updated version with mem leak fix
82101170
AndreasKaratzas [ROCm][RIXL] Updated version with mem leak fix
853a0198
AndreasKaratzas [ROCm][RIXL] Updated version with mem leak fix
c2eae4fd
AndreasKaratzas [ROCm][RIXL] Updated version with mem leak fix
e590ab2c
AndreasKaratzas [ROCm][RIXL] Updated version with mem leak fix
1fabac86
orozery
AndreasKaratzas [ROCm][RIXL] Updated version with mem leak fix
ec839ff4
AndreasKaratzas [ROCm][RIXL] Updated version with mem leak fix
a9e0dc9d
AndreasKaratzas
AndreasKaratzas AndreasKaratzas changed the title [Engine] Fix GPU memory leaks in engine shutdown and add synchronous flush for prefix cache reset [ROCm][Engine] Fix GPU memory leaks in engine shutdown and add synchronous flush for prefix cache reset 230 days ago
AndreasKaratzas [ROCm][RIXL] Updated version with mem leak fix
7a2a3c65
orozery
AndreasKaratzas AndreasKaratzas changed the title [ROCm][Engine] Fix GPU memory leaks in engine shutdown and add synchronous flush for prefix cache reset [ROCm][Engine] Fix GPU memory leaks in engine shutdown and test workaround for async KV prefix cache reset 230 days ago
AndreasKaratzas
orozery
AndreasKaratzas [ROCm][CI] Bump RIXL/UCX hashes and resolve mem leak IPC issue
584c8efa
AndreasKaratzas Merge remote-tracking branch 'origin/main' into akaratza_fix_engine_a…
7d6517c8
AndreasKaratzas [ROCm] Parametrize ROCm path
029b092a
AndreasKaratzas [ROCm] Remove wrong rixl build param
030a7957
AndreasKaratzas Merge remote-tracking branch 'origin/main' into akaratza_fix_engine_a…
cbe0e501
AndreasKaratzas
tjtanaa
AndreasKaratzas
mergify
mergify mergify added needs-rebase
hmellor
AndreasKaratzas
AndreasKaratzas [CI] Fixed small millisecond diff by extending strict and abstract ti…
83a75dae
AndreasKaratzas Merge remote-tracking branch 'origin/main' into akaratza_fix_engine_a…
46ef329f
AndreasKaratzas [CI] Fixed typo in UCX repo link
c85fdb73
AndreasKaratzas AndreasKaratzas requested a review from DarkLight1337 DarkLight1337 223 days ago
AndreasKaratzas AndreasKaratzas requested a review from aarnphm aarnphm 223 days ago
mergify mergify removed needs-rebase
AndreasKaratzas Merge remote-tracking branch 'origin/main' into akaratza_fix_engine_a…
6109039f
SageMoore
SageMoore commented on 2026-04-16
AndreasKaratzas Merge remote-tracking branch 'origin/main' into akaratza_fix_engine_a…
c3ad521d
AndreasKaratzas Restore *in_supports_kw and add sync comment
d64a8182
AndreasKaratzas Merge remote-tracking branch 'origin/main' into akaratza_fix_engine_a…
8926989e
AndreasKaratzas Merge remote-tracking branch 'origin/main' into akaratza_fix_engine_a…
059d6116
AndreasKaratzas Merge remote-tracking branch 'origin/main' into akaratza_fix_engine_a…
f0c67699
tjtanaa
tjtanaa commented on 2026-04-25
AndreasKaratzas Reverted fastsafetensors edit after fix elsewhere
22587966
AndreasKaratzas Reverted fastsafetensors edit after fix elsewhere
e04a8e0d
tjtanaa
tjtanaa approved these changes on 2026-04-25
tjtanaa tjtanaa enabled auto-merge (squash) 210 days ago
tjtanaa tjtanaa merged 95995bbe into main 210 days ago
AndreasKaratzas AndreasKaratzas deleted the akaratza_fix_engine_async_kv branch 210 days ago

Login to write a write a comment.

Login via GitHub

Assignees
No one assigned
Labels
Milestone