vllm
[Feature] Add --rank-tp-ratio for uneven tensor parallelism on mismatched GPUs
#50735

Closed

Commits

[Feature] Add --rank-gpu-id / --rank-gpu-memory-mib for heterogeneous rank-to-GPU mapping

efschu committed 81 days ago
[Feature] Add --rank-tp-ratio for uneven tensor parallelism on mismatched GPUs

efschu committed 81 days ago

Loading