onnxruntime
80bc1d25 - Enable Ep context with external data for CPU nodes (#23498)

Commit
332 days ago
Enable Ep context with external data for CPU nodes (#23498) ### Description When user dump the EP context model, if the nodes not partitioned to the EP, and they have external initializers, then the dumped model still point to the old external data file. It does not make sense that new generated model still point to old external data file. Example, model has node A, B, C, D all has external initializer in ext.bin. So ext.bin contains data for A, B, C, D. After dumping the EP context model, node A is on CPU, node B, C, D are on EP and dumped as EPContext node. If A's data is still in ext.bin, then new generated model has to depend on old ext.bin which contains all external data for the old model which is a big overhead. Fix: For new generated model, user should have option to specify the new external data file, so that the new generated model either pack all initializers into the Onnx model or has all initializers in the external data file. Add option ep.context_model_external_initializers_file_name to specify the new external data file and size threshold. All initializers will be inside the external data fie if the options is specified. Otherwise all initializers will be inside the EP context Onnx model. ### Motivation and Context Fix the issue https://github.com/microsoft/onnxruntime/issues/23358
Author
Parents
Loading