Fix flakiness in the eval integration test (#10008)
I looked at three different logs of devtools tests flaking out on Linux bots on LUCI,
All three logs failed in eval_integration_test.dart:
* Log 1 & Log 2: EvalOnDartLibrary asyncEval supports expressions that do not start with the await keyword timed out after 1 minute (TimeoutException: Test timed out after 1 minutes).
* Log 3: EvalOnDartLibrary asyncEval returns the result of the future completion timed out after 1 minute.
* Cascading failures: On all three runs, following the timeout, the test runner attempted retries (due to retry: 3), but every retry and subsequent test failed with evaluate: (-32000) Service connection disposed.
1. Garbage Collection of reader before evaluation began:
In eval_on_dart_library.dart:390-490, the reader list was previously created in a standalone eval (widgetInspectorService.toId(<dynamic>[], "$readerGroup")). Because WidgetInspectorService only holds objects using
WeakReference, nothing held a strong reference to <dynamic>[] in the target isolate during the network round-trip between DevTools and the target app. If garbage collection occurred in that window, widgetInspectorService.
toObject in the subsequent eval returned null, throwing:
Unhandled exception: type 'Null' is not a subtype of type 'List<dynamic>' in type cast
Because this exception occurred before the try/finally block inside the closure, postEvent("future_completed", ...) was never posted, causing DevTools to hang waiting on future_completed until the test timed out after 1
minute.
2. Pinning loop stopped prematurely:
The pinning loop (while (!isDone && ++bufferTicks <= 20)) intended to pin reader until retrieved. However:
* As soon as isDone was set to true, !isDone became false and the loop terminated immediately (providing zero buffer while postEvent traveled to DevTools and DevTools issued evalInstance). If a GC occurred during that
window, toObject returned null.
* For evaluations taking longer than 1 second (20 ticks of 50ms), ++bufferTicks <= 20 became false while the future was still pending, causing reader to be unpinned before completion.
3. Unhandled error in target isolate:
Any exception occurring during reader initialization or eval setup in the target app closure was unhandled, preventing future_completed from ever firing and causing the target isolate to crash/disconnect.
4. Stale environment reuse on connection drop:
In flutter_test_environment.dart:100-125, _needsSetup was not re-evaluated if the VM service connection dropped (!connectedState.value.connected). When a test timed out and the process disconnected, retries attempted to
reuse the disposed connection, resulting in evaluate: (-32000) Service connection disposed.
1. eval_on_dart_library.dart
* Allocated `final reader = <dynamic>[];` directly inside the evaluated async function and registered it with `widgetInspectorService.toId(reader, "$readerGroup") as String`, eliminating the preliminary eval and ensuring
reader is strongly referenced from the moment of allocation.
* Transmitted reader_id (and any initialization error) directly in postEvent("future_completed", ...).
* Wrapped the entire async function in an outer try/catch that reports errors back via postEvent rather than hanging DevTools.
* Kept reader strongly pinned in the target isolate in the finally block by awaiting in a loop and accessing reader.length until evalInstance calls disposeGroup (or up to 10 seconds timeout).
2. flutter_test_environment.dart
* Added !serviceConnection.serviceManager.connectedState.value.connected to setupEnvironment's re-initialization condition so that if the VM service connection is disposed or dropped, the test environment is
automatically re-created for subsequent tests/retries.
3. eval_integration_test.dart
* Added an explicit test survives garbage collection while the future is pending that triggers full garbage collections in the target isolate via getAllocationProfile(..., gc: true) while the future is pending.
* Removed tags: skipForCustomerTestsTag and retry: 3 now that the flakiness is resolved.
• Ran dart analyze on modified files: No issues found.
• Ran dart format: All modified files formatted.
• Ran 5 consecutive test suites of packages/devtools_app/test/shared/eval_integration_test.dart: All 5 runs passed (5/5 tests passing per run).