Lowering API: Allow for switching to JuliaLowering (#58207)
This PR along with [the associated JuliaLowering
PR](https://github.com/c42f/JuliaLowering.jl/pull/10) allow us to
experiment
with JuliaLowering as the core lowering implementation, mainly for JETLS
development at the moment (cc @aviatesk, @c42f).
## Changes
JuliaLowering works with `JuliaLowering.SyntaxTree` input instead of
Expr. This
change allows`SyntaxTree`s out of parsing and into lowering, hopefully
without
disturbing existing code expecting Expr. The current design wraps
`x::SyntaxTree` like `Expr(:syntaxtree, x)` because otherwise
`toplevel_eval_flex` wouldn't know what `x` is, but once we're past the
experimental phase, it would only make sense for the compiler to
actually know
about the types it's operating on.
### New: `Core._lower`
It (a julia entrypoint), `jl_lower` (C entrypoint), `jl_fl_lower`, and
`fl_lower` all mirror the existing parsing API which was added in
#35243.
<details>
<summary>
Click for pre-PR call graph
</summary>
</details>
After this change:
Like `Core._parse`, `Core._lower` returns an `svec` with the first
element as
the expected result, and the remaining elements can hold extra
information like
the lowered SyntaxTree with provenance info (I've left the specification
of
extra return values for later, when JuliaLowering is more stable and
JETLS knows
what it needs, because we may opt to pack provenance into the lowered
CodeInfo
object instead). Parsing only uses the svec to return an offset, but it
could
actually make use of the the flexibility to return diagnostics in the
future.
It's unfortunately not clear what types we can expect to go into and
come out of
lowering (I got Expr, LineNumberNode, String, Symbol, and Nothing in
testing.)
### Remove `jl_lower_expr_mod`
I realize this isn't being called from C, and Core._lower is a more
sensible
entry point from julia. We could probably also remove some uncalled
parse and
jl_load functions.
## Limitations
These belong to JuliaLowering, but are listed here as known issues for
anyone
who wants to try it out.
- `(module ...)` forms don't go through JuliaLowering yet. Modules are
lowered
to a call to `JuliaLowering.eval_module(parentmod, modname,
syntaxtree)`,
which is currently a stub function that converts to Expr for lowering
and
evaluation, which JuliaLowering can't use.
- Macros work differently. Defining and calling macros should work, but
invoking normal macros defined pre-JuliaLowering will not.
- Compilation is currently slow (~30 seconds for me).
