llama.cpp
553a5c3a - rpc : do not wait for response when sending RPC_CMD_SET_TENSOR (#12943)

Commit
168 days ago
rpc : do not wait for response when sending RPC_CMD_SET_TENSOR (#12943) RPC_CMD_SET_TENSOR always returns an empty response and we send this 4 times per token. We can improve TG speed if we don't wait for this empty response. The performance impact of this change depends on the network latency.
Author
Parents
Loading