vllm-project/vllm

Pull Requests Commits

[Core] Handle skipped warmup and encoder cache budget in extensible KV sizing

njhill committed 16 days ago

9d9592a3

[Core] Simplify extensible KV cache integration

njhill committed 16 days ago

ef630121

[ModelRunner V2] Adapt extensible KV cache to upstream KV layout refactor

njhill committed 16 days ago

df3c39a7

[Doc] register_kv_caches: views are authoritative, not storage nbytes

njhill committed 16 days ago

8e297df8

[Core] Make SimpleCPUOffloadConnector geometry extensible-KV-cache aware

njhill committed 16 days ago

b22f6331

[Core] Tighten packed extensible KV cache invariants

njhill committed 16 days ago

cfeac21f

Fix extensible KV cache lint errors

ZJY0516 committed 16 days ago

26f12922

Fix extensible KV cache connector registrations

ZJY0516 committed 16 days ago

3bf8b9c9

[Core] Defragment extensible KV cache before KV-transfer registration

njhill committed 16 days ago

f1691108

[Core] Allocate shareable (IPC-exportable) VMM memory for KV connectors

njhill committed 16 days ago

b08c4a9a

[ROCm] Add HIP VMM driver backend for the extensible KV cache

njhill committed 16 days ago

14328b9d

[ModelRunner V2] Support KV connectors with extensible KV cache

njhill committed 16 days ago

13338d55

[Core] Support sleep mode with extensible KV cache

njhill committed 16 days ago

6d8ab1fd

[ModelRunner V2] Support packed KV cache layouts with extensible KV cache

njhill committed 16 days ago

f224ae63

[Core] Extensible KV cache: VMM driver probe/fallback, manual size support

njhill committed 16 days ago

de830d3c

[Core] Unify extensible KV cache state on ExtensibleKVCacheBuffers

njhill committed 16 days ago

80c9f8ae

[ModelRunner V2] Support extensible KV cache; size KV from measured warmup memory

njhill committed 16 days ago

450b48e4

[Core] Pick extensible KV cache memory from #47363

njhill committed 16 days ago

5d3350f6

[CI] Initialize fused gated RMSNorm weights (#50377)

AndreasKaratzas committed 16 days ago

Verified bdc98bf5

[Bugfix] Shut down private Tensorizer engines (#49840)

AndreasKaratzas committed 16 days ago

Verified 30e333ca

[Frontend] Preserve bare Inkling text in Python and Rust parsers (#50403)

BugenZhao committed 16 days ago

Verified 629a938a

[ROCm][DSV4] B-preshuffle the attention fp8 projections (#46720)

cagrikymk committed 17 days ago

Verified 12a34a6b

[Build] Fix CUDA release wheel builds (#50243)

khluu committed 17 days ago

Verified 5f8f7286

[Bugfix][MoE] Write Humming results to the supplied output buffer (#50338)

netanel-haber committed 17 days ago

Verified 61cacd27

[DSv4 Perf] Remove redundant full kernel for dsv4, 1.88x kernel performance improvement (#50298)

yewentao256 committed 17 days ago

Verified 837eae64

[DSv4 Perf] Fix redundant memory allocation and copy for dsv4 pp buffer, 448 MiB GPU memory saved (#50312)

yewentao256 committed 17 days ago

Verified 904fae8b

[XPU][CI] skip kimi-k3 test (#50447)

jikunshang committed 17 days ago

Verified f388dd65

[FlexAttention] Avoid encoder block-mask compile explosion (#50339)

AndreasKaratzas committed 17 days ago

Verified 5b958907

[rl] Stateful Trainer Send: IPC [2/N] (#48981)

hao-aaron committed 17 days ago

Verified 30b4e7f4

[Compilation]Fuse Transformers Residual Add + RMSNorm (#48757)

BadrBasowid committed 17 days ago

Verified 59e831c0

Older