nvda

ed5fca6b - Optional experimental support for Microsoft Excel via UI Automation (#12210)

Optional experimental support for Microsoft Excel via UI Automation (#12210) Since 2013, Microsoft has been slowly adding a UI Automation implementation to Microsoft Excel. For quite some time, this implementation was missing too much to be at all useful for NVDA. However, in the last year, Microsoft has increased its effort in this area, starting to fill many of the gaps in the implementation, providing more of a reason for NVDA to start consuming this experimentally, with the goal of one day switching to UI Automation in Excel by default. Not only would this provide significant performance improvemts for NVDA with Excel, but it would also enable scenarios such as using Excel in Application Guard, or remotely via Windows Virtual Desktop, where access to the object model and in-process injection is no longer possible. NVDA will now fall back to relying on UI Automation In Microsoft Excel if available, if NVDA is not able to inject in-process. A new option has been added to NVDA's Advanced settings, which when enabled, forces NVDA to use UI Automation in Microsoft Excel always when available. Although so far navigating, editing and querying cells is all supported, the following features are not yet implemented: • NvDA Elements List for listing formulas, comments etc. The needed UI Automation extensions are not yet available. • Browse mode quick nav. Again, same reason as above. • Setting / consuming screen reader specific column and row headers. This feature probably will not be supported. Instead it is recommended to make these particular sets of data into tables, marking the needed rows and columns as headers in the standrd way for modern Microsoft Excel. As some Excel-specific information on cells does not quite fit with standard NVDA concepts. A new Cell Appearance script (NVDA+o) has been added, which presents a browse mode document, listing these specific bits of information. This script may be changed or removed in future as it is possible to find this information out by looking in the Cell formatting dialog etc anyway, but for now it is there to demonstrate NVDa's ability to fetch this kind of information from UI Automation. Many of Excel's specific features have been exposed by Microsoft through UI automation via either annotations, or custom properties. Standard UIA annotations that NvDA consumes for this support are: • comment: For notes and comments. NVDA exposes the "has comment" state on cells, and reports the comment with a reportComment script (NVDA+alt+c). • Data validation error: the error text is included in the cell's description. • Formula error: the error text is included in the cell's description. • Circular reference error: the error text is included in the cell's description. Microsoft has made use of UI Automation's custom property feature in order to expose many more Excel-specific properties: https://docs.microsoft.com/en-us/office/uia/excel/excelcustomproperties This pr adds an infrastructure in UIAHandler / UIA NVDAObjects which allows registering and accessing these custom property IDs in a standard way. The properties that Microsoft exposes in Excel are: • cellFormula: mapped to NvDA's hasFormula state. • cellNumberFormat: exposed in Cell appearence script (NvDA+o). Note however that Excel currently exposes the raw template value and not the friendly name. So right now this is not as useful as it could be. • hasDataValidation: Exposed in Cell appearence script (NvDA+o) • hasDataValidationDropdown: Mapped to NvDA's submenu state. • dataValidationPrompt: exposed in cell description and Cell appearence script (NvDA+o) • hasConditionalFormatting: exposed in Cell appearence script (NvDA+o) • areGridlinesVisible: exposed in Cell appearence script (NvDA+o) • commentReplyCount: Included in reportComment script (NvDA+alt+c). The above annotations and properties have only been available in Microsoft Excel since build 16.0.13522.10000. Thus turning on this option will only be useful for this build or higher.