pytorch

7c09de83 - [torch deploy] add support for Python C extension modules (#58117)

[torch deploy] add support for Python C extension modules (#58117) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/58117 Previously it was not possible to load C extension modules with deploy because extension modules need to link against the Python.h API functions. Since each libtorchdeploy_interpreter.so had its own copy of these functions, it is not possible to tell dlopen to resolve symbols in a loaded SO from one of these libraries without exposing its symbols globally. This patch adds a custom ELF loader which does the custom loading of attaching c extension libraries to the Python API that loaded the shared library. Simple use of numpy and regex modules appears to work. This diff has some limitations: * 64-bit Linux only. OSX and windows use different formats for shared libraries. 32-bit ELF files are not supported. * debug info is not immediately availiable to debuggers. A script for lldb is provided which can be loaded so that lldb knows about the libraries as they are loaded. * shared libraries can directly use the Python API, but libraries they depend on (via DT_NEEDED entries in their dynamic segment) may not use Python. In the future, we can try to detect whether a sub library uses the Python API and load it with our customer loader. * TLS initialization and library initialization may occur in a different order than what would happen with dlopen, potentially leading to some issues running destructors in TLS segments. Use of this C++ features is relatively rare. Test Plan: Imported from OSS Reviewed By: suo Differential Revision: D28435305 Pulled By: zdevito fbshipit-source-id: 10f046053dd1d250e3c73f2cce8eb945eeba31b6