Fail CI on new linter ecosystem panics (#23597)
Summary
--
The huge number of changes in
https://github.com/astral-sh/ruff/pull/22205#issuecomment-3696660639
should have
obviously been a red flag, but I think it would be nice if CI failed
when new
ecosystem panics were introduced. This PR adds a check for diagnostic
lines that
start with `panic: Panicked at crates/`, raises a `ToolError` if any are
found
in the results from the comparison executable, and then ~~also exits
non-zero if
any errors are returned~~ fails the CI run if the corresponding error
message
was printed.
After trying this out in CI, I opted not to change the script's exit
code itself
because that suppressed the ecosystem comment. It feels a little hackier
this way but preserves the behavior I wanted of both failing CI and
still getting
the ecosystem comment to help with debugging.
Test Plan
--
Local testing on the 0.15.3 tag showing that ruff-ecosystem exited
non-zero and
some manual testing in CI, as you can see below.
