vllm-project/vllm

Pull Requests Commits

Expose HiSparse cache metrics

MatthewBonanni committed 7 days ago

9db784a9

Extract HiSparse metrics from core PR

MatthewBonanni committed 7 days ago

3f91f9bc

Clarify HiSparse component names

MatthewBonanni committed 7 days ago

d64381ad

Fail closed on KV transfer shutdown

MatthewBonanni committed 7 days ago

f4c3344f

Make KV connector step lifecycle explicit

MatthewBonanni committed 7 days ago

5ea267a2

Replace CUDA graph variants with KV residency

MatthewBonanni committed 7 days ago

4a23f3f2

Simplify HiSparse module organization

MatthewBonanni committed 7 days ago

e827b526

MatthewBonanni committed 7 days ago

64ba056b

Support HiSparse speculative decode

MatthewBonanni committed 7 days ago

bbd63f10

Unify sparse MLA metadata contracts

MatthewBonanni committed 7 days ago

ea2b12c8

Centralize HiSparse request-state mapping

MatthewBonanni committed 7 days ago

828ffb27

Clean up HiSparse batch preparation

MatthewBonanni committed 7 days ago

9df35d94

Centralize KV transfer group indexing

MatthewBonanni committed 7 days ago

e88687c5

Keep Mooncake queue accounting unchanged

MatthewBonanni committed 7 days ago

c4d12c30

Fix connector transfer-group handling

MatthewBonanni committed 7 days ago

2653552a

MatthewBonanni committed 7 days ago

b93c7fe0

Centralize KV transfer group selection

MatthewBonanni committed 7 days ago

cde87b88

MatthewBonanni committed 7 days ago

c9a991ff

Merge branch 'main' into feat/hisparse-mla-decode-main

MatthewBonanni committed 7 days ago

9b89539d

Clean up nomenclature

MatthewBonanni committed 7 days ago

3029f064

[Weight processing] Copy over `new_data` attributes in `replace_parameter` (#49601)

fxmarty-amd committed 7 days ago

Verified 81be2e09

[Attention][MLA] Per-request scheduling for MLA chunked context (#50613)

MatthewBonanni committed 7 days ago

Verified b38e111d

Fully generalise input embedding handling in Transformers modelling backend (#51247)

hmellor committed 7 days ago

Verified e7b8d594

MatthewBonanni committed 7 days ago

f0566fa3

[CI] Run basic fullgraph correctness on one GPU (#51271)

mgoin committed 7 days ago

Verified 566c80ed

attn_res kernel latency improvements (#50185)

gnovack committed 7 days ago

Verified 7b4ed496

Update vllm to point to flash-attention commit that builds FA3 with torch stable API. (Retry) (#49599)

cleonard530 committed 7 days ago

Verified 41e7746b

Derive HiSparse GPU page size from backends

MatthewBonanni committed 7 days ago

5a8a59d1

Keep NIXL region metadata in protocol v6

MatthewBonanni committed 7 days ago

31e09263

Support independently sized NIXL cache regions

MatthewBonanni committed 7 days ago

fe2def97

Older