onnxruntime
cced33b2 - Add model compilation in ORT perf test (#25797)

Commit
169 days ago
Add model compilation in ORT perf test (#25797) This PR adds model compilation support in ort perf test * Add `--compile_ep_context` flag in onnxruntime_perf_test. This generates EP context model and prints out compilation time and perf statistics for the compiled model * Prints `Compile time cost` for EP context compilation Sample usage: ```sh $ ./onnxruntime_perf_test -e trt-rtx -I -r 1 "/path/to/model.onnx" --compile_ep_context --compile_model_path "/path/to/model_ctx.onnx" ``` Output: ```sh Compile time cost: 17.8012 s Session creation time cost: 0.966619 s First inference time cost: 8151 ms Total inference time cost: 8.08084 s Total inference requests: 1 Average inference time cost: 8080.84 ms Total inference run time: 8.08085 s Number of inferences per second: 0.123749 Avg CPU usage: 6 % Peak working set size: 7861874688 bytes Avg CPU usage:6 Peak working set size:7861874688 Runs:1 Min Latency: 8.08084 s Max Latency: 8.08084 s P50 Latency: 8.08084 s P90 Latency: 8.08084 s P95 Latency: 8.08084 s P99 Latency: 8.08084 s P999 Latency: 8.08084 s ```
Parents
Loading