onnxruntime
2536acfe - [TRT-EP] Add loadModelProto APIs (#25409)

Commit
153 days ago
[TRT-EP] Add loadModelProto APIs (#25409) ### Description This PR adds three new options for the TRT execution provider: - trt_load_user_initializer - trt_external_data_bytestream - trt_external_data_bytestream_size The idea is to use these options to leverage new TRT 10.13 APIs to give the user more control on how the weights are loaded in the ONNX parser. When `trt_load_user_initializer` is set to true, the EP will own the weights instead of serializing the weights to ModelProto. This reduces overhead in having to serialize large weights. When `trt_external_data_bytestream / trt_external_data_bytestream_size` is provided, the refitEngine() function will be able to read from this bytestream directly to extract weights for the refitter. Also fixes graph_proto_serializer to keep information about external weights. --------- Signed-off-by: Kevin Chen <kevinch@nvidia.com>
Author
Parents
Loading