ruff
dc842db0 - [ty] Detect flaky projects during ecosystem CI jobs (#23178)

Commit
284 days ago
[ty] Detect flaky projects during ecosystem CI jobs (#23178) This brings in an `ecosystem-analyzer` update that detects when we produce flaky results for a project. We do this by running `ty` multiple times against each project, looking for when we produce a different diagnostic at the same file/line/column location. If we do, we consider that diagnostic as flaky. The `diff` and `analyze` HTML reports have been updated to show flakiness information. (A diagnostic changing from "flaky" to "not flaky", or vice versa, is now considered a change, as is a different set of diagnostic codes/messages appearing for a flaky diagnostic.) Since most projects aren't flaky :crossed_fingers:, in the per-PR CI job, we only run flake detection against the projects that we known are flaky. For the others, we still run `ty` once and don't consider any of the diagnostics flaky. In the weekly background job, we run flake detection more aggressively, and on all projects, so that we can update the flaky project list when needed.
Author
Parents
Loading