vllm
[KV Offload] Unified memory layout for offloading workers
#37206
Merged
Overview
Commits
39
Changes
View On GitHub

[KV Offload] Unified memory layout for offloading workers #37206

orozery merged 39 commits into vllm-project:main from omerpaz95:unified_memory
omerpaz95
omerpaz95 First implementation of centralized memory.
4ef57ec6
omerpaz95 Fix 1.
74425110
omerpaz95 omerpaz95 requested a review from ApostaC ApostaC 59 days ago
omerpaz95 omerpaz95 requested a review from orozery orozery 59 days ago
mergify mergify added v1
gemini-code-assist
gemini-code-assist commented on 2026-03-16
orozery
orozery requested changes on 2026-03-17
orozery
omerpaz95 omerpaz95 changed the title Unified memory [KV Offload] Unified memory layout for offloading workers 58 days ago
omerpaz95 Instead of mmap.madvise, explicit pre-fault memory pages.
f24c0651
omerpaz95 SharedMmapRegion in its own file. Fixed hierarchy.
b3be4431
omerpaz95 Firt implementation of interleaved memory layout
02fcc613
mergify
omerpaz95 Fixed Or's code review and commends round 1
ce7c358c
mergify
mergify mergify added needs-rebase
omerpaz95 Before incorporating Or's changes.
176d57f6
omerpaz95 Merge branch 'main' into unified_memory
6ae8ea0d
omerpaz95 Merged with Or's code.
602941b4
omerpaz95 omerpaz95 requested a review from NickLucche NickLucche 50 days ago
mergify mergify added kv-connector
mergify mergify removed needs-rebase
omerpaz95 Deleted duplicate file.
210a8bd9
mergify
mergify mergify added needs-rebase
omerpaz95 Merge branch 'main' into unified_memory
18b850d2
mergify mergify removed needs-rebase
omerpaz95 Fixed segmentation fault.
672ae38c
omerpaz95 pre-rebase.
334e9ee2
omerpaz95 Merge branch 'main' into unified_memory
c3992332
omerpaz95 Pre-testing.
d305ac51
omerpaz95 Some fixes.
1d69e5ae
orozery
orozery requested changes on 2026-04-05
omerpaz95 Fixed Or's comments and some tests.
941366e6
omerpaz95 Fixed assertion in mmap region.
406ea4cf
omerpaz95 Fixed assertion issues and added timing for long ops.
3243af8d
omerpaz95
omerpaz95 Reverted to main.
f4fce527
omerpaz95 Merge branch 'main' into unified_memory
102e9775
omerpaz95 omerpaz95 requested a review from orozery orozery 39 days ago
orozery
orozery
orozery requested changes on 2026-04-06
omerpaz95
omerpaz95 Fixed round of review.
1b907083
omerpaz95 Merge branch 'main' into unified_memory
b8bbb954
omerpaz95 omerpaz95 requested a review from orozery orozery 38 days ago
omerpaz95 Merge branch 'main' into unified_memory
25b91740
orozery
orozery commented on 2026-04-07
omerpaz95 Round #4 for Or's review fixes.
fe5305b6
omerpaz95 Fixed cleanup.
3222c52f
orozery
orozery commented on 2026-04-09
omerpaz95
omerpaz95 Fixed after another round of review. spec.py is now untouched.
bccdf4a5
omerpaz95 Merge branch 'main' into unified_memory
db277af1
orozery
orozery requested changes on 2026-04-13
omerpaz95 Fixed after Ors comments.
020b954e
orozery orozery added ready
orozery
omerpaz95 Fixed comments and tests.
456aa94b
omerpaz95 Merge branch 'main' into unified_memory
a0fda746
omerpaz95 Fixed tests and added proper shutdown to cpu_gpu.
15938a53
omerpaz95 Merge branch 'main' into unified_memory
22fdf55f
omerpaz95 Merge branch 'main' into unified_memory
5f0c542f
orozery
orozery approved these changes on 2026-04-14
orozery Merge branch 'main' into unified_memory
a0379bb2
omerpaz95 Merge branch 'main' into unified_memory
8ced0377
omerpaz95 Merge branch 'main' into unified_memory
649dd485
omerpaz95 Merge branch 'main' into unified_memory
e737d523
omerpaz95 Merge branch 'main' into unified_memory
8adee4ca
orozery orozery merged be0c855e into main 30 days ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
ready v1 kv-connector
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum