llama.cpp
b0fc98e5 - Add a backend interface for collectives across ranks running in separate processes, implement it in the CUDA backend using NCCL, and use it in the RPC server when available.

Commit

4 days ago

Add a backend interface for collectives across ranks running in separate processes, implement it in the CUDA backend using NCCL, and use it in the RPC server when available.

Author

zyang-dev

Committer

ggerganov

Parents

3ab37aed

llama.cpp b0fc98e5 - Add a backend interface for collectives across ranks running in separate processes, implement it in the CUDA backend using NCCL, and use it in the RPC server when available.

llama.cpp
b0fc98e5 - Add a backend interface for collectives across ranks running in separate processes, implement it in the CUDA backend using NCCL, and use it in the RPC server when available.