llvm-project

2566734a - [RISCV] Handle freeze in vp.merge widening codegen prepare (#189346)

[RISCV] Handle freeze in vp.merge widening codegen prepare (#189346) This fixes #189335. Previously we used to widen i1 vp.merges that were used in AnyOf reduction patterns from the loop vectorizer to i8. This improved codegen because RVV doesn't have support for tail undisturbed mask instructions, so i1 vp.merges are expanded into a vid.v + compare sequence. Widening it to i8 allows a single vmerge.vim to be used instead. The pattern used to look like this, where the loop vectorizer emitted a freeze after the reduction: vector.body: ... %3 = call <vscale x 4 x i1> @llvm.vp.merge.nxv4i1(<vscale x 4 x i1> %2, <vscale x 4 x i1> splat (i1 true), <vscale x 4 x i1> %vec.phi, i32 %0) ... br i1 %5, label %middle.block, label %vector.body for.cond.cleanup.loopexit: ; preds = %middle.block %6 = call i1 @llvm.vector.reduce.or.nxv4i1(<vscale x 4 x i1> %3) %7 = freeze i1 %6 br label %for.cond.cleanup But a recent change in InstCombiner now causes the freeze to be hoisted in between the reduction and the vp.merge, breaking the existing pattern: vector.body: ... %3 = call <vscale x 4 x i1> @llvm.vp.merge.nxv4i1(<vscale x 4 x i1> %2, <vscale x 4 x i1> splat (i1 true), <vscale x 4 x i1> %vec.phi, i32 %0) %.fr = freeze <vscale x 4 x i1> %3 ... br i1 %5, label %middle.block, label %vector.body for.cond.cleanup.loopexit: %6 = call i1 @llvm.vector.reduce.or.nxv4i1(<vscale x 4 x i1> %.fr) br label %for.cond.cleanup This patch teaches the transform to handle the freeze. We need to do the change when visiting the freeze instruction otherwise we invalidate the make_early_inc_range iterator.