onnxruntime
655b69ca - [TRT RTX EP] Don't register set device function when we use existing stream (#26542)

Commit
10 days ago
[TRT RTX EP] Don't register set device function when we use existing stream (#26542) ### Description - Don't register set device function when we use existing stream - Fix bug nv_execution_provider.cc : set device only if user did not provide existing stream ### Motivation and Context In some use cases, we push a user generated CUDA context, create streams using this context, and then provide these streams to TRT-RTX. However, we noticed that after calling Run(), the custom context is replaced by another CUDA context created by ORT. This means that TRT-RTX is no longer using the original CUDA context. After investigating further, we found that the new context is being created in onnxruntime/core/framework/stream_execution_context.cc. The solution we propose is to not register set device function if we provide the stream. Also there is a bug in onnxruntime\core\providers\nv_tensorrt_rtx\nv_execution_provider.cc. We should set the device only if the user has not provided any stream. (coherent with the original comment)
Author
Parents
Loading