vllm
[7/N][KV Connector][NIXL] Stage host KV reads through device memory
#53263

Open

[7/N][KV Connector][NIXL] Stage host KV reads through device memory #53263

MatthewBonanni wants to merge 28 commits into feat/hisparse-adaptive-pd-landing from feat/nixl-host-read-staging

MatthewBonanni requested a review from

ApostaC 57 days ago

MatthewBonanni requested a review from

orozery 57 days ago

MatthewBonanni requested a review from

ivanium 57 days ago

MatthewBonanni requested a review from

NickLucche 57 days ago

MatthewBonanni requested a review from

xuechendi 57 days ago

claude commented on 2026-08-21

mergify added kv-connector

MatthewBonanni changed the title ~~feat/nixl host read staging~~ [1/5][KV Connector][NIXL] Stage host KV reads through device memory 57 days ago

MatthewBonanni added ready

LucasWilkinson commented on 2026-08-21

MatthewBonanni changed the title ~~[1/5][KV Connector][NIXL] Stage host KV reads through device memory~~ [1/N][KV Connector][NIXL] Stage host KV reads through device memory 57 days ago

LucasWilkinson commented on 2026-08-21

LucasWilkinson approved these changes on 2026-08-24

NickLucche commented on 2026-08-25

MatthewBonanni force pushed from 0d040344 52 days ago

MatthewBonanni requested a review from

zyongye 52 days ago

MatthewBonanni requested a review from

bigPYJ1151 52 days ago

MatthewBonanni requested a review from

WoosukKwon 52 days ago

MatthewBonanni requested a review from

robertgshaw2-redhat 52 days ago

MatthewBonanni requested a review from

njhill 52 days ago

MatthewBonanni requested a review from

ywang96 52 days ago

MatthewBonanni requested a review from

alexm-redhat 52 days ago

MatthewBonanni requested a review from

heheda12345 52 days ago

MatthewBonanni requested a review from

mgoin 52 days ago

MatthewBonanni requested a review from

tlrmchlsmth 52 days ago

MatthewBonanni requested a review from

yewentao256 52 days ago

MatthewBonanni requested a review from

AndreasKaratzas 52 days ago

MatthewBonanni requested a review from

tdoublep 52 days ago

MatthewBonanni requested a review from

pavanimajety 52 days ago

MatthewBonanni requested a review from

vadiklyutiy 52 days ago

MatthewBonanni requested a review from

zhuohan123 52 days ago

MatthewBonanni requested a review from

youkaichao 52 days ago

MatthewBonanni requested a review from

houseroad 52 days ago

MatthewBonanni requested a review from

ProExpertProg 52 days ago

MatthewBonanni requested a review from

Harry-Chen 52 days ago

mergify added documentation

mergify added ci/build

mergify added deepseek

mergify added minimax

mergify added nvidia

mergify added cpu

mergify added mrv2

mergify added scheduler

mergify added kv-cache-manager

MatthewBonanni changed the base branch from main to feat/hisparse-metrics 52 days ago

MatthewBonanni changed the title ~~[1/N][KV Connector][NIXL] Stage host KV reads through device memory~~ [5/N][KV Connector][NIXL] Stage host KV reads through device memory 52 days ago

MatthewBonanni force pushed to 54d8c008 52 days ago

iyastreb requested changes on 2026-08-26

tvegas1 commented on 2026-08-26

brminich requested changes on 2026-08-26

MatthewBonanni force pushed from 54d8c008 51 days ago

MatthewBonanni force-pushed the feat/hisparse-metrics branch from d331ad01 51 days ago

MatthewBonanni force-pushed the feat/hisparse-metrics branch 51 days ago

MatthewBonanni force pushed 51 days ago

MatthewBonanni force-pushed the feat/hisparse-metrics branch to c200b942 51 days ago

MatthewBonanni force pushed to 4f341703 51 days ago

MatthewBonanni force-pushed the feat/hisparse-metrics branch from c200b942 to 46e3a3df 51 days ago

MatthewBonanni force pushed from 4f341703 to 7ea5ecab 51 days ago

MatthewBonanni force-pushed the feat/hisparse-metrics branch from 46e3a3df to df99b8b5 51 days ago

MatthewBonanni force pushed from 7ea5ecab to a9fa64fa 51 days ago

MatthewBonanni force pushed from a9fa64fa to 59b085ec 50 days ago

MatthewBonanni force-pushed the feat/hisparse-metrics branch to 0fb295a5 50 days ago

MatthewBonanni force-pushed the feat/hisparse-metrics branch from 0fb295a5 50 days ago

MatthewBonanni force pushed from 59b085ec 50 days ago

MatthewBonanni force-pushed the feat/hisparse-metrics branch to 09d7f104 50 days ago

MatthewBonanni force pushed to 52321518 50 days ago

MatthewBonanni force-pushed the feat/hisparse-metrics branch from 09d7f104 to 17d758fe 50 days ago

MatthewBonanni force pushed from 52321518 to 61ad48e1 50 days ago

MatthewBonanni force-pushed the feat/hisparse-metrics branch from 17d758fe to 0fd78c2c 50 days ago

MatthewBonanni force pushed from 61ad48e1 to dfc3f675 50 days ago

MatthewBonanni force-pushed the feat/hisparse-metrics branch from 0fd78c2c 43 days ago

MatthewBonanni force pushed from dfc3f675 43 days ago

MatthewBonanni force-pushed the feat/hisparse-metrics branch to 5769d89d 43 days ago

MatthewBonanni force pushed to 21a05d1b 43 days ago

MatthewBonanni changed the base branch from feat/hisparse-metrics to feat/hisparse-adaptive-pd-landing 42 days ago

MatthewBonanni changed the title ~~[5/N][KV Connector][NIXL] Stage host KV reads through device memory~~ [6/N][KV Connector][NIXL] Stage host KV reads through device memory 42 days ago

MatthewBonanni force-pushed the feat/hisparse-adaptive-pd-landing branch from ddf73127 to 626798bb 42 days ago

MatthewBonanni requested a review from

zou3519 42 days ago

MatthewBonanni requested a review from

BoyuanFeng 42 days ago

mergify added glm

mergify added speculative-decoding

mergify added dflash

mergify added torch.compile

mergify added needs-rebase

MatthewBonanni force pushed from 0fe911e0 to 0c2e53ae 42 days ago

mergify removed needs-rebase

coderabbitai commented on 2026-09-05

MatthewBonanni force-pushed the feat/hisparse-adaptive-pd-landing branch from 626798bb to d221370c 41 days ago

MatthewBonanni force pushed from d0ce57a8 to 111d49da 41 days ago

coderabbitai commented on 2026-09-06

MatthewBonanni force-pushed the feat/hisparse-adaptive-pd-landing branch from d221370c to d09f851f 40 days ago

MatthewBonanni force pushed from 111d49da to abb4b1f1 40 days ago

MatthewBonanni force pushed from abb4b1f1 to 52ed0111 40 days ago

MatthewBonanni force-pushed the feat/hisparse-adaptive-pd-landing branch from d09f851f to 59bcd4ce 40 days ago

MatthewBonanni force-pushed the feat/hisparse-adaptive-pd-landing branch from 59bcd4ce to 27a0d411 39 days ago

MatthewBonanni force pushed from 52ed0111 to 16a8a033 39 days ago

MatthewBonanni changed the title ~~[6/N][KV Connector][NIXL] Stage host KV reads through device memory~~ [7/N][KV Connector][NIXL] Stage host KV reads through device memory 39 days ago

MatthewBonanni force-pushed the feat/hisparse-adaptive-pd-landing branch from 27a0d411 39 days ago

MatthewBonanni force pushed from 16a8a033 39 days ago

MatthewBonanni force-pushed the feat/hisparse-adaptive-pd-landing branch 39 days ago

MatthewBonanni force pushed 39 days ago

MatthewBonanni force-pushed the feat/hisparse-adaptive-pd-landing branch 39 days ago

MatthewBonanni force pushed 39 days ago

MatthewBonanni force-pushed the feat/hisparse-adaptive-pd-landing branch to 5ae789f3 38 days ago

MatthewBonanni force pushed to b13fe038 38 days ago

MatthewBonanni force-pushed the feat/hisparse-adaptive-pd-landing branch from 5ae789f3 38 days ago

MatthewBonanni force pushed from b13fe038 38 days ago

MatthewBonanni force pushed to c7fb9ed1 38 days ago

MatthewBonanni force-pushed the feat/hisparse-adaptive-pd-landing branch to bd218eb2 38 days ago

MatthewBonanni force-pushed the feat/hisparse-adaptive-pd-landing branch from bd218eb2 to 378cc310 36 days ago

MatthewBonanni force pushed from c7fb9ed1 to 0b5b87f8 36 days ago

[KV Connector] Stage host-memory KV reads through device memory

8331192c

Harden NIXL host-staging lifecycle

ef0211bf

Rename host read stager for destination semantics

c709453e

Let host stager own its shutdown lifecycle

22a75868

Remove implementation-mirroring shutdown test

fb275414

Relax NIXL shutdown executor assertion

80ed180a

[NIXL] Simplify host-stager progress and shutdown (#53354)

6e223384

Do not suppress NIXL shutdown errors

e12f1d14

Harden NIXL host staging initialization

edf1b7ec

Revert "Harden NIXL host staging initialization"

fd526b2b

Trim NIXL host staging test scaffolding

32a32a0b

review comments

32ed8d72

Preserve staged transfer failure lifecycle

34694b12

Rebase NIXL host staging onto adaptive landing

53045536

Fix HiSparse NIXL host pool ownership

aa4e5072

Prepare HiSparse row mappings before forward

73a3a1e8

Stage mixed HiSparse host imports through GPU

bfec845e

Gate NIXL host write staging to CUDA

96a71a16

Preserve empty NIXL receive completion

fb0ddee5

Keep empty NIXL reads on notify-only path

e9d452d9

Update mixed receive fixture for host staging

a0a7be07

Simplify staged receive failure coordination

ebd1b871

Preserve host-staging receive contract

5152456b

Fix staged NIXL transfer completion mode

a5d57e2f

Report staged NIXL receive failures promptly

d5a8e4e8

Format restacked NIXL transfer completion path

2a5fbec9

Preserve receive draining before reporting staged NIXL failures

90e6fb03

LucasWilkinson force-pushed the feat/hisparse-adaptive-pd-landing branch from 378cc310 to ec2d4ab3 35 days ago

LucasWilkinson force pushed from 0b5b87f8 to 90e6fb03 35 days ago

Merge branch 'feat/hisparse-adaptive-pd-landing' into feat/nixl-host-…

e574f4af

mergify added needs-rebase

Reviewers

LucasWilkinson

brminich

iyastreb

coderabbitai

NickLucche

claude

tvegas1

ApostaC

orozery

ivanium

xuechendi

zyongye

bigPYJ1151

WoosukKwon

robertgshaw2-redhat

njhill

ywang96

alexm-redhat

heheda12345

mgoin

tlrmchlsmth

yewentao256

AndreasKaratzas

tdoublep

pavanimajety

vadiklyutiy

zhuohan123

youkaichao

houseroad

ProExpertProg

Harry-Chen

zou3519

BoyuanFeng

Assignees

No one assigned

Labels

documentation speculative-decoding ready torch.compile needs-rebase ci/build deepseek cpu kv-connector nvidia mrv2 dflash scheduler kv-cache-manager glm minimax

Milestone

No milestone

vllm [7/N][KV Connector][NIXL] Stage host KV reads through device memory #53263 Open

[7/N][KV Connector][NIXL] Stage host KV reads through device memory #53263

vllm
[7/N][KV Connector][NIXL] Stage host KV reads through device memory
#53263

Open