Downgrade to Python 3.7 due to stack corruption in Python 3.8+ (#12298)
Fixes #12152
Fixes #12154
Since upgrading to Python 3.8, several serious crashes in NVDA have been reported. Specifically:
• NVDA crashing when using the SAPI4 speech synthesizer: #12152
• NVDA crashing when using Windows Explorer on Windows Server 2012: #12154
Both of these issues were traced to stack corruption after a Python callback of NVDA's was called from external libraries. In SAPI4's case, after calling NVDA's implementation of ITTSBufNotifySink::TextDataStarted, and in the Windows Server 2012 case: IUIAutomationPropertyChangedEventHandler::handlePropertyChangedEvent.
It seems as though libFFI / Python ctypes is not cleaning the stack properly after executing a Python callback with the stdcall calling convention (ctypes WINFUNCTYPE), where the callback contained at least one argument larger than 4 bytes (E.g. a long long, or a VARIANT struct), and the arguments preceding it were such that this argument was not aligned to an 8 byte boundary. E.g. the callback might be:
• callback(void*, long long) or
• callback(void*, void*, int, VARIANT)
See Python bug: https://bugs.python.org/issue38748
On that bug I have attached a minimal testcase.
This bug affects both Python 3.8 and Python 3.9.
The bug is most likely in the libFFI project used by Python's ctypes module. Python 3.8 switched to a much more recent and official version of libFFI I believe.
Although we do really want to move to Python 3.8+ as soon as we can, right now this bug makes it impossible to do so. We could specifically work around the currently known manifestations by moving some of that code into C++, but that brings its own risks, and we still don't know where else this issue may appear in our code. The appropriate thing to do right now is stay on Python 3.7 until we can work with Python / libFFI to get this resolved.
Description of how this pull request fixes the issue:
Downgrades to Python 3.7 by referencing Python 3.7 (rather than 3.8) in NVDA's build system.
The existing PRs that needed to be reverted were:
• Updating brlAPI to a Python 3.8 build: nvaccess/nvda-misc-deps#20
• Switching to using Python's own pgettext: #12109
• calling os.add_dll_directory when loading liblouis: #12020
None of these PRs provided any user visible changes.
The rest of the Python 3.8 work, including the switch to a virtual
environment etc is all compatible with Python 3.7 and can remain.