llvm-project

7e526f7b - [mlir][affine] emit `in_bounds` on `transfer_read`/`write` when statically provable in `affine-super-vectorize` (#201180)

[mlir][affine] emit `in_bounds` on `transfer_read`/`write` when statically provable in `affine-super-vectorize` (#201180) This patch fixes an issue reported on the MLIR Discourse ([May 2026](https://discourse.llvm.org/t/mlir-affine-affine-super-vectorize-does-not-set-in-bounds-on-transfer-ops-for-statically-divisible-shapes/90785/3)), that I also came across during a study I reported in [my blog post](https://federicobruzzone.github.io/posts/mlir-study.html). `affine-super-vectorize` always creates `vector.transfer_read` and `vector.transfer_write` without an `in_bounds` attribute, even when it is statically provable that every access stays within bounds. This forces downstream lowering to unconditionally emit `llvm.intr.masked.load`/`llvm.intr.masked.store`: masked intrinsics that carry $\sim3\times$ overhead on AArch64/NEON and prevent auto-vectorization (see the MLIR Discourse). ## Root cause `vectorizeAffineLoad` and `vectorizeAffineStore` in `SuperVectorize.cpp` forwarded neither the `in_bounds` mask nor any analysis of it when constructing the transfer ops. The fix computes the mask at vectorization time by inspecting the permutation map and the memref type via a new `computeInBoundsMask` helper: - `AffineDimExpr`: the vector dimension maps to a concrete memref dimension. If that dimension is static and divisible by the vector width, the accesses are guaranteed in-bounds. - `AffineConstantExpr`: a broadcast (the dimension is collapsed to a constant index). A broadcast can never be out-of-bounds. - Everything else (e.g., AffineAddExpr on dynamic dimensions): conservatively left false. <details> <summary>Reproduction</summary> Run the lowering pipeline on any static-size affine copy loop: ```mlir // file: copy.mlir func.func @copy(%A: memref<512x512xf32>, %B: memref<512x512xf32>) { affine.for %i = 0 to 512 { affine.for %j = 0 to 512 { %v = affine.load %A[%i, %j] : memref<512x512xf32> affine.store %v, %B[%i, %j] : memref<512x512xf32> } } return } ``` ``` mlir-opt copy.mlir \ --affine-super-vectorize="virtual-vector-size=4" \ --convert-vector-to-llvm \ --finalize-memref-to-llvm \ --convert-func-to-llvm ``` Before this patch: masked intrinsics despite fully static, divisible dimensions: ``` %31 = llvm.intr.masked.load %30, %25, %16 {alignment = 4 : i32} : (!llvm.ptr, vector<4xi1>, vector<4xf32>) -> vector<4xf32> llvm.intr.masked.store %31, %43, %38 {alignment = 4 : i32} : vector<4xf32>, vector<4xi1> into !llvm.ptr ``` After this patch: plain vector load/store: ``` %23 = llvm.load %22 {alignment = 4 : i64} : !llvm.ptr -> vector<4xf32> llvm.store %23, %28 {alignment = 4 : i64} : vector<4xf32>, !llvm.ptr ``` </details> AI Disclaimer: I used AI for the tests. --------- Signed-off-by: Federico Bruzzone <federico.bruzzone.i@gmail.com> Co-authored-by: Artem Gindinson <gindinson@roofline.ai>