onnxruntime
1cb4307d - Retain Python async run resources (#32041)

Commit
9 days ago
Retain Python async run resources (#32041) This pull request updates the ONNX Runtime Python bindings to improve memory management and API flexibility for asynchronous inference sessions. The main changes ensure that Python objects used as inputs to `run_async` are properly referenced and not prematurely garbage collected, and that the API is more Pythonic by accepting `py::object` for session and run options. It also adds a test to verify that input arrays are kept alive during asynchronous execution. **Improvements to memory management and API flexibility:** * The `AsyncResource` struct now stores references to Python feed objects, the session, and run options to ensure they are kept alive for the duration of async execution (`onnxruntime/python/onnxruntime_pybind_state.cc`). * The `run_async` binding now accepts `py::object` for the session and run options, and internally casts them as needed. This makes the API more Pythonic and flexible (`onnxruntime/python/onnxruntime_pybind_state.cc`). * When preparing feeds for `run_async`, the code now stores each input object in `feed_objects` before creating the corresponding `OrtValue`, ensuring the Python objects are not garbage collected too early (`onnxruntime/python/onnxruntime_pybind_state.cc`). **Testing and validation:** * A test using `weakref` was added to verify that input arrays passed to `run_async` remain alive until the callback is invoked, preventing premature garbage collection (`onnxruntime/test/python/onnxruntime_test_python.py`) [[1]](diffhunk://#diff-bc2d3954a8ed883e3036dc7a675c7ba13b6299e1079728e7b6069cab1f49479cR690) [[2]](diffhunk://#diff-bc2d3954a8ed883e3036dc7a675c7ba13b6299e1079728e7b6069cab1f49479cR702) [[3]](diffhunk://#diff-bc2d3954a8ed883e3036dc7a675c7ba13b6299e1079728e7b6069cab1f49479cL712-R721). * The test also exercises explicit deletion and garbage collection of inputs, session, and run options to ensure robustness (`onnxruntime/test/python/onnxruntime_test_python.py`).
Author
Parents
Loading