vllm-project/vllm

Pull Requests Commits

Keep a fixed non-torch reserve under the torch memory cap

njhill committed 1 day ago

355c79ae

Measure the warmup transient before CUDA graph capture

njhill committed 1 day ago

1d27e25d

set torch mem limit

njhill committed 1 day ago

f75424d1

inline core method

njhill committed 1 day ago

b2b3abb5

simplify startup cross-rank agreement

njhill committed 1 day ago

879cfe88

simplify warmup

njhill committed 1 day ago

10cc6fb9

simplify collective

njhill committed 1 day ago

9dd5989b

njhill committed 1 day ago

0865885b

njhill committed 1 day ago

e72a93cd

njhill committed 1 day ago

82b09cce

simplify gpu_worker.py changes

njhill committed 1 day ago

df268c2f

handle weight reloading

njhill committed 1 day ago

fa42f1d6

test and warmup fixes

njhill committed 1 day ago

d568e4c4

njhill committed 1 day ago

264a5184

fix dp config alignment

njhill committed 1 day ago

8e8748e5

njhill committed 1 day ago

56d3c83f

[Core] Extensible KV cache: adapt to the SimpleCPUOffload block-count handshake and engine-side DCP interleave

njhill committed 1 day ago

a3d8337f

[Core] Extensible KV cache: fix CI failures and gate connectors that cannot share it

njhill committed 1 day ago

c770cadc

[Core] Extensible KV cache: probe the HIP set-access defect and load the runtime torch uses

njhill committed 1 day ago

c0cd41eb

[Core] Extensible KV cache: adapt to Elastic EP on MRV2 and externally held weights

njhill committed 1 day ago

fab8e3fe

[Core] Extensible KV cache: ask the deferred connector for its memory pool

njhill committed 1 day ago

27dd74e7

[Core] Extensible KV cache: treat a connector's custom memory pool as unsupported

njhill committed 1 day ago

03518f90

[Core] Extensible KV cache: fix CI regressions from optional utilization and pin the dummy sampler passes

njhill committed 1 day ago

44e06a37

[Core] Extensible KV cache: simplify the engine-side committable block agreement

njhill committed 1 day ago

bc34364b

[Core] Extensible KV cache: agree the committable block count in the engine, not by all-reduce

njhill committed 1 day ago

6ba706b4

[Core] Extensible KV cache: review nits

njhill committed 1 day ago

4e25defe

[Core] Extensible KV cache: granule-aligned segments under KV connectors and exact fit sizing

njhill committed 1 day ago

e3a97276

[Core] Extensible KV cache: agree the committable block count across ranks

njhill committed 1 day ago

b2ea0d1d

[Core] Extensible KV cache: startup correctness fixes from review

njhill committed 1 day ago

bff328a1

[Core] Extensible KV cache: warm up a full-budget prefill and keep its reserved peak free

njhill committed 1 day ago

4a979569

Older