num_kv_heads` deployments (#19691)

Commit

1 year ago

[P/D][NixlConnector] Support `tp_size > num_kv_heads` deployments (#19691) Signed-off-by: NickLucche <nlucches@redhat.com> Co-authored-by: Nick Hill <nhill@redhat.com>

References

#19691 - [P/D][NixlConnector] Support `tp_size > num_kv_heads` deployments

Author

NickLucche

Parents

f17aec0d

vllm 2ebff5b7 - [P/D][NixlConnector] Support `tp_size > num_kv_heads` deployments (#19691)

vllm
2ebff5b7 - [P/D][NixlConnector] Support `tp_size > num_kv_heads` deployments (#19691)