[LV] Build all scalar steps for non-uniform induction variables

When building the steps for scalar induction variables, we previously attempted
to determine if all the scalar users of the induction variable were uniform. If
they were, we would only emit the step corresponding to vector lane zero. This
optimization was too aggressive. We generally don't know the entire set of
induction variable users that will be scalar. We have
isScalarAfterVectorization, but this is only a conservative estimate of the
instructions that will be scalarized. Thus, an induction variable may have
scalar users that aren't already known to be scalar. To avoid emitting unused
steps, we can only check that the induction variable is uniform. This should
fix PR30542.

Reference: https://llvm.org/bugs/show_bug.cgi?id=30542
llvm-svn: 282863

1 files changed, 3 insertions, 14 deletions

diff --git a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
index 897d14a1404..c2282705313 100644
--- a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
+++ b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
@@ -2326,22 +2326,11 @@ void InnerLoopVectorizer::buildScalarSteps(Value *ScalarIV, Value *Step,
   assert(ScalarIVTy->isIntegerTy() && ScalarIVTy == Step->getType() &&
          "Val and Step should have the same integer type");
 
-  auto scalarUserIsUniform = [&](User *U) -> bool {
-    auto *I = cast<Instruction>(U);
-    return !OrigLoop->contains(I) || !Legal->isScalarAfterVectorization(I) ||
-           Legal->isUniformAfterVectorization(I);
-  };
-
   // Determine the number of scalars we need to generate for each unroll
-  // iteration. If EntryVal is uniform or all it's scalar users are uniform, we
-  // only need to generate the first lane. Otherwise, we generate all VF
-  // values. We are essentially determining if the induction variable has no
-  // "multi-scalar" (non-uniform scalar) users.
+  // iteration. If EntryVal is uniform, we only need to generate the first
+  // lane. Otherwise, we generate all VF values.
   unsigned Lanes =
-      Legal->isUniformAfterVectorization(cast<Instruction>(EntryVal)) ||
-              all_of(EntryVal->users(), scalarUserIsUniform)
-          ? 1
-          : VF;
+      Legal->isUniformAfterVectorization(cast<Instruction>(EntryVal)) ? 1 : VF;
 
   // Compute the scalar steps and save the results in VectorLoopValueMap.
   ScalarParts Entry(UF);