vllm
Enable Cross layers KV cache layout at NIXL Connector
#30207

Merged

Enable Cross layers KV cache layout at NIXL Connector #30207

NickLucche merged 62 commits into vllm-project:main from liranschour:nixl_kv_cont_cross_layers

liranschour requested a review from

NickLucche 153 days ago

liranschour requested a review from

ApostaC 153 days ago

mergify added kv-connector

gemini-code-assist commented on 2025-12-07

chatgpt-codex-connector commented on 2025-12-07

Cross layers implementation

5195265c

Fix linting

0f368880

liranschour force pushed to 0f368880 153 days ago

mergify added v1

liranschour force pushed to 0f368880 152 days ago

Add cross layers compatibility check

8d36b4ba

orozery requested changes on 2025-12-10

Move cross_layers logic into TpKVTopology

2a20197c

liranschour force pushed to 2a20197c 150 days ago

NickLucche commented on 2025-12-11

Code review minor fix

073b30e0

liranschour force pushed to 073b30e0 149 days ago

Linting...

b403a9e0

liranschour force pushed to b403a9e0 146 days ago

liranschour requested a review from

orozery 146 days ago

orozery requested changes on 2025-12-15

liranschour force pushed 144 days ago

liranschour force pushed 143 days ago

liranschour requested a review from

orozery 143 days ago

liranschour force pushed 143 days ago

Code review fixes

06d31842

liranschour force pushed to 06d31842 143 days ago

orozery requested changes on 2025-12-17

mergify added needs-rebase

Update vllm/distributed/kv_transfer/kv_connector/utils.py

cd278668

Update vllm/distributed/kv_transfer/kv_connector/utils.py

19319af6

Update vllm/distributed/kv_transfer/kv_connector/v1/nixl_connector.py

994bf1de

Update vllm/distributed/kv_transfer/kv_connector/v1/nixl_connector.py

0efeba3b

Update vllm/distributed/kv_transfer/kv_connector/v1/nixl_connector.py

ef8e7adf

Code review fixes

6e2b7512

orozery requested changes on 2025-12-22

Code review fix

5e66e8f6

liranschour requested a review from

orozery 133 days ago

orozery commented on 2025-12-28

Code review fix

eaf5e3d4

liranschour requested a review from

orozery 133 days ago

Code review fix

e85f458a

orozery commented on 2025-12-28

Merge remote-tracking branch 'vllm/main' into nixl_kv_cont_cross_layers

fff09357

Code review fix

9bd95989

mergify removed needs-rebase

orozery approved these changes on 2025-12-28

NickLucche added ready

mergify added needs-rebase

Merge remote-tracking branch 'vllm/main' into nixl_kv_cont_cross_layers

cd57ed8c

n/a

f153e839

n/a

15f2a78a

n/a

0cb18250

n/a

9630c8ea

n/a

c148f6de

n/a

96329f6f

Merge remote-tracking branch 'vllm/main' into nixl_kv_cont_cross_layers

0394b36c

mergify removed needs-rebase

NickLucche commented on 2026-01-14

cursor commented on 2026-01-14

liranschour force pushed to 0394b36c 115 days ago

Unit test fix

b4d70450

Unit test fix

52b11557

Unit test fix

e34db34d

Unit test fix

edc07557

n/a

03af3ece

n/a

f0f2cf9d

Merge branch 'main' into nixl_kv_cont_cross_layers

c3e1e5ee

orozery commented on 2026-01-18

Code review fix

701d4efd

Code review fix

e7df5f83

Code review fix

6dae9b52

Handle hetrogenous TP for FLASHINFER and TRITON

012bb9e8

n/a

99b34019

n/a

9fe2eb6b

n/a

043c4d8f

Run cross layers only for FlashAttention and FLASHINFER

fe7197ce

Enhance test_register_kv_caches

5d59ea68

Documentation

392e5d59

liranschour force pushed to 392e5d59 108 days ago

mergify added documentation

Merge branch 'main' into nixl_kv_cont_cross_layers

d0e9aedc

Code review fix

3c6921f5

orozery commented on 2026-01-21

Code review fix

ed3180cb

Code review fix

ced9ad45

Code review fix

580dbc4c

orozery commented on 2026-01-21

Code review fix

19fff296

liranschour force pushed to 19fff296 108 days ago

n/a

dd97e99b

n/a

fce2050e

n/a

71611509

n/a

4d3890ec

n/a

d92bf969

n/a

6991cdda

n/a

2791d34f

n/a

ff1f2440

Add cross layers blocks to run_accuracy_tests.sh

92f26280

NickLucche approved these changes on 2026-01-21

Code review fix

4715cedb

Code review fix

c2e0ca05

Merge branch 'main' into nixl_kv_cont_cross_layers

7d1df767

Minor fix

d9ad710c

NickLucche commented on 2026-01-22

NickLucche enabled auto-merge (squash) 108 days ago

NickLucche merged 64e3d67a into main 108 days ago

Reviewers

NickLucche

orozery

cursor

chatgpt-codex-connector

gemini-code-assist

ApostaC

Assignees

No one assigned

Labels

documentation ready v1 kv-connector

Milestone

No milestone

vllm Enable Cross layers KV cache layout at NIXL Connector #30207 Merged

Enable Cross layers KV cache layout at NIXL Connector #30207

vllm
Enable Cross layers KV cache layout at NIXL Connector
#30207

Merged