Make Compiler tests runnable as package (#56632)
Makes `test Compiler` work properly (as in use the Compiler package, not
Base.Compiler) and pass tests, but still needs to be made parallel in a
follow-on.
---------
Co-authored-by: Shuhei Kadowaki <40514306+aviatesk@users.noreply.github.com>
Co-authored-by: Kristoffer Carlsson <kcarlsson89@gmail.com>
Co-authored-by: Shuhei Kadowaki <aviatesk@gmail.com>
