[REPLCompletions] async tab/hint completion
REPL utilizes inference for smarter completions, but since its inference
engine is independent from the native cache and issues like #52763 in
`REPLInterpreter`'s inference, sometimes delays can be noticeable during
tab/hint-completion, and REPL can even freeze in the worst cases.
To address this, the commit allows tab/hint-completion to run
asynchronously on a `:default` thread. This approach removes delays in
multi-threaded process and prevents REPL hang in extreme cases like #52763.
In detail, for tab-completion, completion task simply runs on a separate
thread since completion itself does not block anything. For
hint-completion however, considering that a stuttering display of hints
can be bothersome, when the completion task doesn't conclude within
0.01 sec, we cancel the hint-completion to keep the input in the REPL
smooth.
There are still some points left open for discussion:
- Is it better to allocate a separate thread specifically for REPL
completion? I'm not sure if this is even possible though.
- Should we make this an optional feature, considering the concern some
users might have about reduced thread resources?
- For situations where completion hangs or is slow, like in #52763,
canceling the completion task to release resources would be ideal.
However, my understanding is that there's no safe method to cancel a
task, is it correct?
- At present, this commit `Threads.@spawn`s at a relatively high level
in the completion pipeline. But it might be more effective to
`Threads.@spawn` closer to the point where inference is actually
invoked, such as in `complete_line` or `repl_eval_ex`.
