vllm-project/vllm

Pull Requests Commits

Keep scheduler alive for pending HiSparse spills

MatthewBonanni committed 2 days ago

9a21608f

Cap HiSparse resident pages during full-ISL admission

MatthewBonanni committed 2 days ago

267d86d4

Update HiSparse replay test for masked MHA arguments

MatthewBonanni committed 3 days ago

69be17a4

Adapt HiSparse to current attention initialization

MatthewBonanni committed 3 days ago

de6b8af9

Expose GSM8K request timeout

MatthewBonanni committed 3 days ago

b81be4a4

Refresh HiSparse state during graph replay

MatthewBonanni committed 3 days ago

d2c7a178

Fix HiSparse mixed single-token warmup

MatthewBonanni committed 3 days ago

8e3b2c06

Release HiSparse cache after CUDA graph profiling

MatthewBonanni committed 3 days ago

fde7f9d9

[HiSparse] Compile index conversion warmup with fake tensors

MatthewBonanni committed 3 days ago

fb064565

[HiSparse] Decouple attention warmup from model dummy runs

MatthewBonanni committed 3 days ago

388ad340

Replace HiSparse invalidation Triton kernel

MatthewBonanni committed 3 days ago

0fb8762e

[HiSparse] Import Triton runtime for cache invalidation

MatthewBonanni committed 3 days ago

162b7e42

[HiSparse] Remove decode routing mirror test

MatthewBonanni committed 3 days ago

be5f78a2

[HiSparse] Compose with adaptive speculative decoding

MatthewBonanni committed 3 days ago

b41c8b3d

Initialize HiSparse through connector registration

MatthewBonanni committed 3 days ago

86cd0d3c

Move HiSparse connector beside KV connectors

MatthewBonanni committed 3 days ago

34ca94c1

Fix NIXL push worker shutdown signature

MatthewBonanni committed 3 days ago

d7ede865

Align HiSparse with KV connector lifecycle

MatthewBonanni committed 3 days ago

46aae4df

Compose HiSparse with generic indexer offloading

MatthewBonanni committed 3 days ago

19aacd9e

Fix slot mapping for HiSparse hot cache groups

MatthewBonanni committed 3 days ago

6271c1ef

Add HiSparse host-resident sparse MLA decode

MatthewBonanni committed 3 days ago

1ff8d763

Support per-region NIXL transfer geometry

MatthewBonanni committed 3 days ago

Verified ac096a1f

[kernel] Integrate FlashInfer BF16 CuTeDSL Low Latency GEMM (#50572)

jiahanc committed 3 days ago

Verified 7c877062

Add support for loading dflash2 model in speculators format (#53797)

fynnsu committed 3 days ago

Verified f79a2f55

[Bugfix][MRV2] Isolate sleep-mode KV allocations (#53508)

Ronald1995 committed 3 days ago

Verified 479eeb32

Upgrade tpu-inference to v0.28.0 (#54020)

meiyeh123 committed 3 days ago

Verified d262964e

[Attention] Enable masked MHA for GLM-5 head dimensions (#53785)

MatthewBonanni committed 3 days ago

Verified de9250ac

[Bugfix][Parser] Scope reasoning-end detection to the current turn via turn-boundary tokens (#54089)

Xarbirus committed 3 days ago

Verified 3a9bfc20

[Perf][DSv4] Use native CUDA SwiGLU clamp kernel for Humming MoE (throughput +1.40%) (#53685)

chaunceyjiang committed 3 days ago

Verified 7d5769b2

[Kimi Perf] Tune hopper low latency gemm kernel, 4%~97% performance improvement (#54088)

yewentao256 committed 3 days ago

Verified 9818bb3d

Older