[mlir][IR] Add builtin `TokenType` (#195640)
Introduces a new parameterless, opaque, builtin SSA value type,
`!token`. A token cannot appear in a value-forwarding position such as
`cf.br`, `arith.select`, `scf.for` iter-args or function call/return.
Walking back from any token use reaches the producing operation without
crossing such a boundary. Tokens carry no runtime data and cannot
constant-fold.
This contract is enforced by changing the default ODS `AnyType`
predicate to exclude tokens. In addition, token-producing and
token-consuming ops must opt in by implementing the `ProducesToken` and
`ConsumesToken` traits. The IR verifier checks for the presence of these
traits.
Note: CSE, DCE, hoisting, and `OperationEquivalence` remain unchanged. A
stronger contract (e.g., uniqueness, arity, paired lifetime), if
desirable based on the semantics of token-producing and token-consuming
ops, can be expressed at the op level via existing mechanisms (side
effects, block arguments, traits, attributes), keeping the type
orthogonal to operation-level transformations. This mirrors LLVM's
`token` type.
Other changes:
* Builtin bytecode: new entry for `TokenType`.
* AsmParser / AsmPrinter: `token` keyword for the textual format.
* Remove `LLVMTokenType` from the LLVM dialect and use the builtin
`TokenType` instead.
* Async dialect: references to `async::TokenType` are qualified to
disambiguate from the new builtin `TokenType`.
* Documentation: new `mlir/docs/Tokens.md` covering the structural
contract, design rationale, and ODS integration.
No changes to `Operation`, the generic op syntax, the bytecode op
encoding, or core C++ APIs around `Operation`.
This commit is in preparation of adding support for breaking exit from
regions. (E.g., early exit from loops.)
RFC:
https://discourse.llvm.org/t/rfc-add-a-builtin-token-type-to-mlir/90706
Assisted-by: claude-opus-4.7-thinking-high
---------
Co-authored-by: Mehdi Amini <joker.eph@gmail.com>
