vllm
[7/N][KV Connector][NIXL] Stage host KV reads through device memory
#53263

Open

[7/N][KV Connector][NIXL] Stage host KV reads through device memory #53263

MatthewBonanni wants to merge 28 commits into feat/hisparse-adaptive-pd-landing from feat/nixl-host-read-staging

MatthewBonanni requested a review from

ApostaC 57 days ago

MatthewBonanni requested a review from

orozery 57 days ago

MatthewBonanni requested a review from

ivanium 57 days ago

MatthewBonanni requested a review from

NickLucche 57 days ago

MatthewBonanni requested a review from

xuechendi 57 days ago

claude commented on 2026-08-21

mergify added kv-connector

MatthewBonanni changed the title ~~feat/nixl host read staging~~ [1/5][KV Connector][NIXL] Stage host KV reads through device memory 57 days ago

MatthewBonanni added ready

LucasWilkinson commented on 2026-08-21

MatthewBonanni changed the title ~~[1/5][KV Connector][NIXL] Stage host KV reads through device memory~~ [1/N][KV Connector][NIXL] Stage host KV reads through device memory 57 days ago

LucasWilkinson commented on 2026-08-21

LucasWilkinson approved these changes on 2026-08-24

NickLucche commented on 2026-08-25

MatthewBonanni force pushed from 0d040344 53 days ago

MatthewBonanni requested a review from

zyongye 53 days ago

MatthewBonanni requested a review from

bigPYJ1151 53 days ago

MatthewBonanni requested a review from

WoosukKwon 53 days ago

MatthewBonanni requested a review from

robertgshaw2-redhat 53 days ago

MatthewBonanni requested a review from

njhill 53 days ago

MatthewBonanni requested a review from

ywang96 53 days ago

MatthewBonanni requested a review from

alexm-redhat 53 days ago

MatthewBonanni requested a review from

heheda12345 53 days ago

MatthewBonanni requested a review from

mgoin 53 days ago

MatthewBonanni requested a review from

tlrmchlsmth 53 days ago

MatthewBonanni requested a review from

yewentao256 53 days ago

MatthewBonanni requested a review from

AndreasKaratzas 53 days ago

MatthewBonanni requested a review from

tdoublep 53 days ago

MatthewBonanni requested a review from

pavanimajety 53 days ago

MatthewBonanni requested a review from

vadiklyutiy 53 days ago

MatthewBonanni requested a review from

zhuohan123 53 days ago

MatthewBonanni requested a review from

youkaichao 53 days ago

MatthewBonanni requested a review from

houseroad 53 days ago

MatthewBonanni requested a review from

ProExpertProg 53 days ago

MatthewBonanni requested a review from

Harry-Chen 53 days ago

mergify added documentation

mergify added ci/build

mergify added deepseek

mergify added minimax

mergify added nvidia

mergify added cpu

mergify added mrv2

mergify added scheduler

mergify added kv-cache-manager

MatthewBonanni changed the base branch from main to feat/hisparse-metrics 53 days ago

MatthewBonanni changed the title ~~[1/N][KV Connector][NIXL] Stage host KV reads through device memory~~ [5/N][KV Connector][NIXL] Stage host KV reads through device memory 53 days ago

MatthewBonanni force pushed to 54d8c008 53 days ago

iyastreb requested changes on 2026-08-26

tvegas1 commented on 2026-08-26

brminich requested changes on 2026-08-26

MatthewBonanni force pushed from 54d8c008 52 days ago

MatthewBonanni force-pushed the feat/hisparse-metrics branch from d331ad01 52 days ago

MatthewBonanni force-pushed the feat/hisparse-metrics branch 52 days ago

MatthewBonanni force pushed 52 days ago

MatthewBonanni force-pushed the feat/hisparse-metrics branch to c200b942 52 days ago

MatthewBonanni force pushed to 4f341703 52 days ago

MatthewBonanni force-pushed the feat/hisparse-metrics branch from c200b942 to 46e3a3df 51 days ago

MatthewBonanni force pushed from 4f341703 to 7ea5ecab 51 days ago

MatthewBonanni force-pushed the feat/hisparse-metrics branch from 46e3a3df to df99b8b5 51 days ago

MatthewBonanni force pushed from 7ea5ecab to a9fa64fa 51 days ago

MatthewBonanni force pushed from a9fa64fa to 59b085ec 51 days ago

MatthewBonanni force-pushed the feat/hisparse-metrics branch to 0fb295a5 51 days ago

MatthewBonanni force-pushed the feat/hisparse-metrics branch from 0fb295a5 51 days ago

MatthewBonanni force pushed from 59b085ec 51 days ago

MatthewBonanni force-pushed the feat/hisparse-metrics branch to 09d7f104 51 days ago

MatthewBonanni force pushed to 52321518 51 days ago

MatthewBonanni force-pushed the feat/hisparse-metrics branch from 09d7f104 to 17d758fe 51 days ago

MatthewBonanni force pushed from 52321518 to 61ad48e1 51 days ago

MatthewBonanni force-pushed the feat/hisparse-metrics branch from 17d758fe to 0fd78c2c 51 days ago

MatthewBonanni force pushed from 61ad48e1 to dfc3f675 51 days ago

MatthewBonanni force-pushed the feat/hisparse-metrics branch from 0fd78c2c 44 days ago

MatthewBonanni force pushed from dfc3f675 44 days ago

MatthewBonanni force-pushed the feat/hisparse-metrics branch to 5769d89d 44 days ago

MatthewBonanni force pushed to 21a05d1b 44 days ago

MatthewBonanni changed the base branch from feat/hisparse-metrics to feat/hisparse-adaptive-pd-landing 43 days ago

MatthewBonanni changed the title ~~[5/N][KV Connector][NIXL] Stage host KV reads through device memory~~ [6/N][KV Connector][NIXL] Stage host KV reads through device memory 43 days ago

MatthewBonanni force-pushed the feat/hisparse-adaptive-pd-landing branch from ddf73127 to 626798bb 43 days ago

MatthewBonanni requested a review from

zou3519 43 days ago

MatthewBonanni requested a review from

BoyuanFeng 43 days ago

mergify added glm

mergify added speculative-decoding

mergify added dflash

mergify added torch.compile

mergify added needs-rebase

MatthewBonanni force pushed from 0fe911e0 to 0c2e53ae 43 days ago

mergify removed needs-rebase

coderabbitai commented on 2026-09-05

MatthewBonanni force-pushed the feat/hisparse-adaptive-pd-landing branch from 626798bb to d221370c 41 days ago

MatthewBonanni force pushed from d0ce57a8 to 111d49da 41 days ago

coderabbitai commented on 2026-09-06

MatthewBonanni force-pushed the feat/hisparse-adaptive-pd-landing branch from d221370c to d09f851f 41 days ago

MatthewBonanni force pushed from 111d49da to abb4b1f1 41 days ago

MatthewBonanni force pushed from abb4b1f1 to 52ed0111 40 days ago

MatthewBonanni force-pushed the feat/hisparse-adaptive-pd-landing branch from d09f851f to 59bcd4ce 40 days ago

MatthewBonanni force-pushed the feat/hisparse-adaptive-pd-landing branch from 59bcd4ce to 27a0d411 39 days ago

MatthewBonanni force pushed from 52ed0111 to 16a8a033 39 days ago

MatthewBonanni changed the title ~~[6/N][KV Connector][NIXL] Stage host KV reads through device memory~~ [7/N][KV Connector][NIXL] Stage host KV reads through device memory 39 days ago

MatthewBonanni force-pushed the feat/hisparse-adaptive-pd-landing branch from 27a0d411 39 days ago

MatthewBonanni force pushed from 16a8a033 39 days ago

MatthewBonanni force-pushed the feat/hisparse-adaptive-pd-landing branch 39 days ago

MatthewBonanni force pushed 39 days ago

MatthewBonanni force-pushed the feat/hisparse-adaptive-pd-landing branch 39 days ago

MatthewBonanni force pushed 39 days ago

MatthewBonanni force-pushed the feat/hisparse-adaptive-pd-landing branch to 5ae789f3 39 days ago

MatthewBonanni force pushed to b13fe038 39 days ago

MatthewBonanni force-pushed the feat/hisparse-adaptive-pd-landing branch from 5ae789f3 39 days ago

MatthewBonanni force pushed from b13fe038 39 days ago

MatthewBonanni force pushed to c7fb9ed1 39 days ago

MatthewBonanni force-pushed the feat/hisparse-adaptive-pd-landing branch to bd218eb2 39 days ago

MatthewBonanni force-pushed the feat/hisparse-adaptive-pd-landing branch from bd218eb2 to 378cc310 36 days ago

MatthewBonanni force pushed from c7fb9ed1 to 0b5b87f8 36 days ago

[KV Connector] Stage host-memory KV reads through device memory

8331192c

Harden NIXL host-staging lifecycle

ef0211bf

Rename host read stager for destination semantics

c709453e

Let host stager own its shutdown lifecycle

22a75868

Remove implementation-mirroring shutdown test

fb275414

Relax NIXL shutdown executor assertion

80ed180a

[NIXL] Simplify host-stager progress and shutdown (#53354)

6e223384

Do not suppress NIXL shutdown errors

e12f1d14

Harden NIXL host staging initialization

edf1b7ec

Revert "Harden NIXL host staging initialization"

fd526b2b

Trim NIXL host staging test scaffolding

32a32a0b

review comments

32ed8d72

Preserve staged transfer failure lifecycle

34694b12

Rebase NIXL host staging onto adaptive landing

53045536

Fix HiSparse NIXL host pool ownership

aa4e5072

Prepare HiSparse row mappings before forward

73a3a1e8

Stage mixed HiSparse host imports through GPU

bfec845e

Gate NIXL host write staging to CUDA

96a71a16

Preserve empty NIXL receive completion

fb0ddee5

Keep empty NIXL reads on notify-only path

e9d452d9

Update mixed receive fixture for host staging

a0a7be07

Simplify staged receive failure coordination

ebd1b871

Preserve host-staging receive contract

5152456b

Fix staged NIXL transfer completion mode

a5d57e2f

Report staged NIXL receive failures promptly

d5a8e4e8

Format restacked NIXL transfer completion path

2a5fbec9

Preserve receive draining before reporting staged NIXL failures

90e6fb03

LucasWilkinson force-pushed the feat/hisparse-adaptive-pd-landing branch from 378cc310 to ec2d4ab3 36 days ago

LucasWilkinson force pushed from 0b5b87f8 to 90e6fb03 36 days ago

Merge branch 'feat/hisparse-adaptive-pd-landing' into feat/nixl-host-…

e574f4af

mergify added needs-rebase

Reviewers

LucasWilkinson

brminich

iyastreb

coderabbitai

NickLucche

claude

tvegas1

ApostaC

orozery

ivanium

xuechendi

zyongye

bigPYJ1151

WoosukKwon

robertgshaw2-redhat

njhill

ywang96

alexm-redhat

heheda12345

mgoin

tlrmchlsmth

yewentao256

AndreasKaratzas

tdoublep

pavanimajety

vadiklyutiy

zhuohan123

youkaichao

houseroad

ProExpertProg

Harry-Chen

zou3519

BoyuanFeng

Assignees

No one assigned

Labels

documentation speculative-decoding ready torch.compile needs-rebase ci/build deepseek cpu kv-connector nvidia mrv2 dflash scheduler kv-cache-manager glm minimax

Milestone

No milestone

vllm [7/N][KV Connector][NIXL] Stage host KV reads through device memory #53263 Open

[7/N][KV Connector][NIXL] Stage host KV reads through device memory #53263

vllm
[7/N][KV Connector][NIXL] Stage host KV reads through device memory
#53263

Open