llama.cpp
6df4032d - rpc : typed request helpers and a single validation step on the server

Commit
1 day ago
rpc : typed request helpers and a single validation step on the server Client and server share one table (rpc_msg_types) that maps each fixed-size command to its request and response structs. The client sends them with dispatcher->call<CMD>() and call_async<CMD>(), the server serves them with rpc_serve<CMD>(), so the per-command code in rpc_serve_client and at the call sites is gone and a wrong struct for a command does not compile. SET_TENSOR, SET_TENSOR_2D and GRAPH_COMPUTE get header structs, written with rpc_writer and parsed with a bounds-checked rpc_reader. The wire format is unchanged. Validation of client data is now in one place: - deserialize_tensor rejects what ggml_new_tensor would assert on (op range, ne[0] not a multiple of the block size, element count overflow) and computes the tensor size without overflow before the buffer check - rpc_check_region replaces the per-handler bounds checks and checks the request against the tensor size, which ggml_backend_tensor_set/get/memset assert on, so an in-buffer but out-of-tensor request no longer aborts - graph_compute builds the graph in one pass over the tensors, which the client sends in topological order: no recursion, and unknown or duplicate ids, duplicate nodes, sources sent after their users, compute nodes with unallocated operands and views outside their view_src buffer are rejected; a tensor may be its own source (ggml_cast) - GET_ALLOC_SIZE fails if a source tensor is invalid - a client-sized allocation that fails closes the connection instead of terminating the server
Author
Parents
Loading