onnxruntime
[TensorRT EP] Fix InferenceSession::Run() not thread-safe issue
#19301

Merged

[TensorRT EP] Fix InferenceSession::Run() not thread-safe issue #19301

chilo-ms merged 10 commits into main from chi/trt_ep_concurrent_fix

update

dd2b9158

modify comment

7b697257

lintrunner -a

5ec9e392

chilo-ms requested a review from

jywu-msft 2 years ago

skip cudaStreamSynchronize when cuda graph is enabled

1f068712

move sync at the end of inference run

4a950ffd

update

413d5068

put sync after enqueueV3 and make it configurable

c3ff278c

chilo-ms marked this pull request as ready for review 2 years ago

chilo-ms added release:1.17.0

skip cudaStreamSynchronize when cuda graph is enabled

b4926eb8

jywu-msft commented on 2024-01-29

refactor

f7d3a5a4

jywu-msft dismissed these changes on 2024-01-29

external stream won't have to sync explicitly

d29ef3df

chilo-ms dismissed their stale review via d29ef3df 2 years ago

jywu-msft approved these changes on 2024-01-30

chilo-ms merged 00d04812 into main 2 years ago

chilo-ms deleted the chi/trt_ep_concurrent_fix branch 2 years ago

snnn removed release:1.17.0

Reviewers

jywu-msft

Assignees

No one assigned

Labels

None yet

Milestone

No milestone