onnxruntime
Add QNN EP HTP shared memory allocator
#23136
Merged
Overview
Commits
61
Changes
View On GitHub

Add QNN EP HTP shared memory allocator #23136

edgchen1 merged 61 commits into main from edgchen1/qnn_ep_rpcmem
edgchen1
edgchen1 save work
110a3bca
edgchen1 save work
0ba3a2fb
edgchen1 add logging for setting QNN tensor memory, update comment
8436b14a
edgchen1 add option to enable HTP shared memory allocator to onnxruntime_perf_…
c9826f44
edgchen1 hack - try to cache mem handles in QnnModel
c07c35e5
edgchen1 Remove duplicate include.
60dc8374
edgchen1 hack, continued - move cache out to SharedContext
24e072f0
edgchen1 Merge remote-tracking branch 'origin/main' into edgchen1/qnn_ep_rpcmem
e66cbef2
edgchen1 move mem handle registration to allocator
8c515dab
edgchen1 hook up some test code
18e2780b
edgchen1 Merge remote-tracking branch 'origin/main' into edgchen1/qnn_ep_rpcmem
09ddce59
edgchen1 rename to RpcMemAllocator to HtpSharedMemoryAllocator
a65bb71c
edgchen1 Merge remote-tracking branch 'origin/main' into edgchen1/qnn_ep_rpcmem
bfb135ed
edgchen1 remove onnx protobuf dependency from allocator.h, add shared provider…
f179a0d8
edgchen1 remove unused CPUAllocator::TensorAlloc declaration
7645ef45
baijumeswani Check for nullptr when trying to free
10437328
edgchen1 move mem handle management to QNN backend manager
022f4bcb
edgchen1 remove IAllocator::TensorAlloc()
c527dee2
edgchen1 document IAllocator::Free
e4f72b36
edgchen1 remove IAllocator__TensorAlloc
39ff9012
edgchen1 Merge remote-tracking branch 'origin/main' into edgchen1/qnn_ep_rpcmem
1bed5a45
edgchen1 fix android build warning
d70db84e
edgchen1 remove shared mem handles from shared context
45ef8837
edgchen1 remove allocation clean up callback removal, use weak_ptrs in allocat…
d2e7b3c5
edgchen1 some clean up
c892c18e
edgchen1 more clean up
b295eef0
edgchen1 add helper to get qnn error message
13f5e308
edgchen1 use make_shared for QnnBackendManager
d5eace13
edgchen1 add test to qnn_basic_test.cc, document allocator parameter.
bacbcdc1
edgchen1 Merge remote-tracking branch 'origin/main' into edgchen1/qnn_ep_rpcmem
30cd9edb
edgchen1 rename variables
b29ab610
edgchen1 revert changes to onnxruntime/test/providers/qnn/max_min_op_test.cc
67a54b89
github-actions
github-actions commented on 2024-12-18
github-advanced-security
github-advanced-security commented on 2024-12-18
jywu-msft jywu-msft requested a review from HectorSVC HectorSVC 1 year ago
edgchen1 fix formatting
c0569e22
edgchen1 skip test if not android and not windows
dd45c84b
edgchen1 update comment
959d8df0
edgchen1
edgchen1 commented on 2024-12-19
edgchen1
edgchen1 commented on 2024-12-19
edgchen1 remove QnnBackendManager::ReleaseQnnContextMemHandles declaration, up…
ab48516b
edgchen1 edgchen1 requested a review from skottmckay skottmckay 1 year ago
edgchen1 edgchen1 requested a review from baijumeswani baijumeswani 1 year ago
edgchen1 edgchen1 requested a review from adrianlizarraga adrianlizarraga 1 year ago
edgchen1 edgchen1 requested a review from jywu-msft jywu-msft 1 year ago
edgchen1 add onnxruntime_c_api.h include to ortmemoryinfo.h
4a3f6c39
edgchen1 Merge remote-tracking branch 'origin/main' into edgchen1/qnn_ep_rpcmem
65ce4b1a
edgchen1 rename GetQnnTensorDataSize to GetQnnTensorDataSizeInBytes
ff125413
edgchen1 add QnnBackendManager::Create function to ensure shared_ptr usage
5e6e1039
edgchen1 make some QnnBackendManager member functions private, update comment
78e86cc7
edgchen1 edgchen1 marked this pull request as ready for review 357 days ago
edgchen1 edgchen1 changed the title [WIP] Add QNN EP HTP shared memory allocator Add QNN EP HTP shared memory allocator 357 days ago
edgchen1 document GetOrRegister functions
e665a2b8
HectorSVC HectorSVC added ep:QNN
edgchen1 add enable_htp_shared_memory_allocator to available_keys
425023b2
edgchen1 Merge remote-tracking branch 'origin/main' into edgchen1/qnn_ep_rpcmem
781a4a02
edgchen1
edgchen1 commented on 2025-01-09
edgchen1 make DlError return const char*
4d292081
HectorSVC
HectorSVC commented on 2025-01-09
HectorSVC
HectorSVC commented on 2025-01-09
skottmckay
skottmckay commented on 2025-01-09
edgchen1
edgchen1 commented on 2025-01-09
edgchen1 Use ORT_DISALLOW_COPY_ASSIGNMENT_AND_MOVE for SharedContext
568c9a73
edgchen1 use safeint instead of manually checking against int max
8b955358
edgchen1 add/update doc for enable_htp_shared_memory_allocator option
515999c2
edgchen1 formatting
6986839e
edgchen1 add some comments about HtpSharedmemoryAllocator impl
00b286b9
skottmckay
skottmckay commented on 2025-01-09
baijumeswani
baijumeswani commented on 2025-01-09
edgchen1 initialize with QNN_MEM_DESRIPTOR_INIT
88dec645
edgchen1 address comments
4ca3ea75
edgchen1 rework context handle ownership
7a88c3f4
edgchen1 add / update tests
f3730353
edgchen1 add check for qnn tensor dynamic shape
e86ff2eb
edgchen1 Add comment about multi-threading considerations
6fa33f0b
edgchen1 fix test comment
4101cca6
edgchen1 fix formatting
14af7ad6
edgchen1
edgchen1 commented on 2025-01-11
edgchen1 add ifdef to use htp backend if on arm64 or linux.
2f5c93c9
edgchen1 Merge remote-tracking branch 'origin/main' into edgchen1/qnn_ep_rpcmem
b868a9f6
yuslepukhin
yuslepukhin commented on 2025-01-13
edgchen1 fix typo
7ca45523
yuslepukhin
yuslepukhin
yuslepukhin commented on 2025-01-13
yuslepukhin
yuslepukhin commented on 2025-01-13
yuslepukhin
yuslepukhin requested changes on 2025-01-13
edgchen1
edgchen1 edgchen1 requested a review from yuslepukhin yuslepukhin 350 days ago
yuslepukhin
yuslepukhin approved these changes on 2025-01-13
baijumeswani
baijumeswani approved these changes on 2025-01-13
skottmckay
skottmckay approved these changes on 2025-01-13
adrianlizarraga
adrianlizarraga approved these changes on 2025-01-14
adrianlizarraga
adrianlizarraga commented on 2025-01-14
edgchen1
azure-pipelines
edgchen1 edgchen1 merged 04030f64 into main 349 days ago
edgchen1 edgchen1 deleted the edgchen1/qnn_ep_rpcmem branch 349 days ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
ep:QNN
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum