System test robustness (PR #13983)
* Dump speech / devInfo / braille from main thread
These require access to IAccessible or RPC, which must happen from
main thread.
* Easier test selection for devs
- When "--include NVDA" is present, it isn't possible to
select some other tag only, tests matching either tag will be run.
- This results in many tests running when a developer may only wish to run chrome, or symbols tests.
- The initial intention was to prevent developers accidentally running installer tests.
- Now achieved by supplying a tag that matches no tests.
* Ensure hwnd vals are comparable
- HWND passed into EnumWindowsProc
was a lpclong, other HWND vals were integer objects,
comparisons for the same HWND were always false.
* Fix chrome process tracking
- Keep the 'start' process around until chrome exits (/wait)
Track this independently of the chrome process
- Look up the chrome window handle and title and save to instance var on
the library instance. Work with "Window" class instances rather than HWND
directly.
- Use the start process to confirm exit, logging shows that the start
process still exists prior to sending control w, but not after.
* Don't use kiosk mode for chrome
So that screenshots can contain more information.
* Don't use win32 to focus chrome.
The test unable to set the foreground window with user32.SetForegroundWindow
Testing locally shows that this won't work, documentation backs this up.
The RF test runner process is not the foreground process.
See docs: https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-setforegroundwindow#remarks
* Wait for start marker
Adjustments to how the tests check for the start marker.
Now:
- move focus to the address bar explicitly (for a known starting point)
- move the focus to the document with an explicit chrome shortcut
- report the current line
* Toggle chrome focus
Introduces an optional de/re-focus of chrome (currently disabled).
This may help to work-around issues with NVDA missing foreground events / not injecting.
- view desktop
- alt+tab back to chrome
- wait for chrome to be foreground app
* Don't exit chrome after each test
- Initially this was done to ensure a "clean slate" for each subsequent test.
- However, each test sample loads in a new tab already, the risk of interference between tabs is low.
- Exiting the tab / chrome is disabled on the off-chance that the action of closing and opening
chrome causes delays, or causes chrome to run background updates, or other actions which may
interfere with the tests on appveyor,
- Additionally, for developers, it is easier to inspect test samples after the test runs if they are left
open which may be important if the test fails.
- However, this leaves many tabs open for the developer to tidy up.
* Add note about postmortem debugger
The debug_break command can be caught by a postmortem debugger.
* Give robot remote server more time to start up
- Appveyor systems seem to have become slower.
- Some the amount of time it takes for NVDA to start up is approaching 10 seconds.
- Add logging to be able to inspect the timing of various steps to start NVDA during system tests.
