vllm
[Core] Support disaggregated prefill with Mooncake Transfer Engine
#10884
Merged
Merged
[Core] Support disaggregated prefill with Mooncake Transfer Engine #10884
KuntaiDu
merged
25 commits into
vllm-project:main
from
kvcache-ai:upstream-mooncake-integration
Rebase from main to work with PR 10502.
d52dbc80
Update format of mooncake config ValueError.
c8e9d07a
Modify metadata transfer logic to support tp.
b718f1e5
Fix format to make ruff happy.
08e2800f
Add instructions when mooncake is not installed.
81797467
Merge branch 'main' into upstream-mooncake-integration
ba82d71b
Merge branch 'main' into upstream-mooncake-integration
76d484c8
fix import order to make isort happy.
e912055c
Fix format to make yapf happy.
2396f01d
Add solution for ports conflict on the same node.
31514a03
Fix format to make mypy happy.
2ef10be5
Get head_size and num_heads from model config to address bugs on Volt…
0823e47d
Add support for other metadata server backend.
6fb95fb3
Change code to align with PR 11058.
a5758b1b
Fix typo.
33e44556
Merge branch 'main' into upstream-mooncake-integration
f3312b9a
Reuse simple connector for mooncake pipe.
343c4741
Remove mooncake connector.
eaa1a451
fix isort.
83e4db91
fix mypy.
e8ee5c2f
move PyNcclPipe import to fix mypy.
bc01eae6
still trying to fix mypy.
b45ff65b
fix typo and fix mypy.
aeccf4fb
trying to fix mypy again.
8c2135ae
remove unused kvpipe base.
875ca4cb
KuntaiDu
approved these changes
on 2024-12-15
Login to write a write a comment.
Login via GitHub