diff options
| author | Hans Wennborg <hans@chromium.org> | 2019-11-07 11:00:02 +0100 |
|---|---|---|
| committer | Hans Wennborg <hans@chromium.org> | 2019-11-07 11:00:02 +0100 |
| commit | eaff3004019f97c64c88ab76da6b25106b659b30 (patch) | |
| tree | 06181ea12fc4d48e9bb74f34b4336c826ec067f3 /llvm/lib/Analysis | |
| parent | c5e4cf40ac459aae996180089a9831959ceb3d05 (diff) | |
| download | bcm5719-llvm-eaff3004019f97c64c88ab76da6b25106b659b30.tar.gz bcm5719-llvm-eaff3004019f97c64c88ab76da6b25106b659b30.zip | |
Revert f0c2a5a "[LV] Generalize conditions for sinking instrs for first order recurrences."
It broke Chromium, causing "Instruction does not dominate all uses!" errors.
See https://bugs.chromium.org/p/chromium/issues/detail?id=1022297#c1 for a
reproducer.
> If the recurrence PHI node has a single user, we can sink any
> instruction without side effects, given that all users are dominated by
> the instruction computing the incoming value of the next iteration
> ('Previous'). We can sink instructions that may cause traps, because
> that only causes the trap to occur later, but not on any new paths.
>
> With the relaxed check, we also have to make sure that we do not have a
> direct cycle (meaning PHI user == 'Previous), which indicates a
> reduction relation, which potentially gets missed by
> ReductionDescriptor.
>
> As follow-ups, we can also sink stores, iff they do not alias with
> other instructions we move them across and we could also support sinking
> chains of instructions and multiple users of the PHI.
>
> Fixes PR43398.
>
> Reviewers: hsaito, dcaballe, Ayal, rengolin
>
> Reviewed By: Ayal
>
> Differential Revision: https://reviews.llvm.org/D69228
Diffstat (limited to 'llvm/lib/Analysis')
| -rw-r--r-- | llvm/lib/Analysis/IVDescriptors.cpp | 40 |
1 files changed, 14 insertions, 26 deletions
diff --git a/llvm/lib/Analysis/IVDescriptors.cpp b/llvm/lib/Analysis/IVDescriptors.cpp index bdebd71d7f6..6fb600114bc 100644 --- a/llvm/lib/Analysis/IVDescriptors.cpp +++ b/llvm/lib/Analysis/IVDescriptors.cpp @@ -699,37 +699,25 @@ bool RecurrenceDescriptor::isFirstOrderRecurrence( // Ensure every user of the phi node is dominated by the previous value. // The dominance requirement ensures the loop vectorizer will not need to // vectorize the initial value prior to the first iteration of the loop. - // TODO: Consider extending this sinking to handle memory instructions and - // phis with multiple users. - - // Returns true, if all users of I are dominated by DominatedBy. - auto allUsesDominatedBy = [DT](Instruction *I, Instruction *DominatedBy) { - return all_of(I->uses(), [DT, DominatedBy](Use &U) { - return DT->dominates(DominatedBy, U); - }); - }; - + // TODO: Consider extending this sinking to handle other kinds of instructions + // and expressions, beyond sinking a single cast past Previous. if (Phi->hasOneUse()) { - Instruction *I = Phi->user_back(); - - // If the user of the PHI is also the incoming value, we potentially have a - // reduction and which cannot be handled by sinking. - if (Previous == I) - return false; - - if (DT->dominates(Previous, I)) // We already are good w/o sinking. - return true; - - // We can sink any instruction without side effects, as long as all users - // are dominated by the instruction we are sinking after. - if (I->getParent() == Phi->getParent() && !I->mayHaveSideEffects() && - allUsesDominatedBy(I, Previous)) { - SinkAfter[I] = Previous; + auto *I = Phi->user_back(); + if (I->isCast() && (I->getParent() == Phi->getParent()) && I->hasOneUse() && + DT->dominates(Previous, I->user_back())) { + if (!DT->dominates(Previous, I)) // Otherwise we're good w/o sinking. + SinkAfter[I] = Previous; return true; } } - return allUsesDominatedBy(Phi, Previous); + for (User *U : Phi->users()) + if (auto *I = dyn_cast<Instruction>(U)) { + if (!DT->dominates(Previous, I)) + return false; + } + + return true; } /// This function returns the identity element (or neutral element) for |

