summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms/Utils
Commit message (Collapse)AuthorAgeFilesLines
...
* [BasicBlockUtils] Check for unreachable preds before updating LI in ↵Anna Thomas2018-01-021-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | UpdateAnalysisInformation Summary: We are incorrectly updating the LI when loop-simplify generates dedicated exit blocks for a loop. The issue is that there's an implicit assumption that the Preds passed into UpdateAnalysisInformation are reachable. However, this is not true and breaks LI by incorrectly updating the header of a loop. One such case is when we generate dedicated exits when the exit block is a landing pad (through SplitLandingPadPredecessors). There maybe other cases as well, since we do not guarantee that Preds passed in are reachable basic blocks. The added test case shows how loop-simplify breaks LI for the outer loop (and DT in turn) after we try to generate the LoopSimplifyForm. Reviewers: davide, chandlerc, sanjoy Reviewed By: davide Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41519 llvm-svn: 321653
* [SimplifyCFG] Return to the pass manager the correct value.Davide Italiano2017-12-311-1/+1
| | | | | | | I wanted to commit this with r321603, but I failed to squash the two commits. llvm-svn: 321606
* [Utils/Local] Use `auto` when the type is obvious. NFCI.Davide Italiano2017-12-311-6/+6
| | | | llvm-svn: 321605
* [Utils] Remove commented debug message. NFCI.Davide Italiano2017-12-311-4/+0
| | | | llvm-svn: 321604
* [SimplifyCFG] Stop hoisting musttail calls incorrectly.Davide Italiano2017-12-311-0/+11
| | | | | | PR35774. llvm-svn: 321603
* Use phi ranges to simplify code. No functionality change intended.Benjamin Kramer2017-12-309-173/+112
| | | | llvm-svn: 321585
* Revert r321377, it causes regression to https://reviews.llvm.org/P8055.Guozhi Wei2017-12-281-179/+0
| | | | llvm-svn: 321528
* [Unroll][DebugInfo] Propagate loop body's debug location to epilog preheaderZhaoshi Zheng2017-12-261-1/+6
| | | | | | | NewExit and epilog PreHeader should has the same debug loc as the original loop body, instead of original loop exit. llvm-svn: 321465
* Make helpers static. No functionality change.Benjamin Kramer2017-12-241-3/+4
| | | | llvm-svn: 321425
* [SimplifyCFG] Don't do if-conversion if there is a long dependence chainGuozhi Wei2017-12-221-0/+179
| | | | | | | | | | If after if-conversion, most of the instructions in this new BB construct a long and slow dependence chain, it may be slower than cmp/branch, even if the branch has a high miss rate, because the control dependence is transformed into data dependence, and control dependence can be speculated, and thus, the second part can execute in parallel with the first part on modern OOO processor. This patch checks for the long dependence chain, and give up if-conversion if find one. Differential Revision: https://reviews.llvm.org/D39352 llvm-svn: 321377
* Add hasProfileData() to check if a function has profile data. NFC.Easwaran Raman2017-12-221-1/+1
| | | | | | | | | | | | | | | | | | | Summary: This replaces calls to getEntryCount().hasValue() with hasProfileData that does the same thing. This refactoring is useful to do before adding synthetic function entry counts but also a useful cleanup IMO even otherwise. I have used hasProfileData instead of hasRealProfileData as David had earlier suggested since I think profile implies "real" and I use the phrase "synthetic entry count" and not "synthetic profile count" but I am fine calling it hasRealProfileData if you prefer. Reviewers: davidxl, silvas Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41461 llvm-svn: 321331
* [SimplifyCFG] Avoid quadratic on a predecessors number behavior in ↵Michael Zolotukhin2017-12-211-14/+10
| | | | | | | | | | | | | | | | | | | | | | instruction sinking. If a block has N predecessors, then the current algorithm will try to sink common code to this block N times (whenever we visit a predecessor). Every attempt to sink the common code includes going through all predecessors, so the complexity of the algorithm becomes O(N^2). With this patch we try to sink common code only when we visit the block itself. With this, the complexity goes down to O(N). As a side effect, the moment the code is sunk is slightly different than before (the order of simplifications has been changed), that's why I had to adjust two tests (note that neither of the tests is supposed to test SimplifyCFG): * test/CodeGen/AArch64/arm64-jumptable.ll - changes in this test mimic the changes that previous implementation of SimplifyCFG would do. * test/CodeGen/ARM/avoid-cpsr-rmw.ll - in this test I disabled common code sinking by a command line flag. llvm-svn: 321236
* [ICP] Expose unconditional call promotion interfaceMatthew Simpson2017-12-201-77/+178
| | | | | | | | | | | | | | | | | | | | | | | This patch modifies the indirect call promotion utilities by exposing and using an unconditional call promotion interface. The unconditional promotion interface (i.e., call promotion without creating an if-then-else) can be used if it's known that an indirect call has only one possible callee. The existing conditional promotion interface uses this unconditional interface to promote an indirect call after it has been versioned and placed within the "then" block. A consequence of unconditional promotion is that the fix-up operations for phi nodes in the normal destination of invoke instructions are changed. This is necessary because the existing implementation assumed that an invoke had been versioned, creating a "merge" block where a return value bitcast could be placed. In the new implementation, the edge between a promoted invoke's parent block and its normal destination is split if needed to add a bitcast for the return value. If the invoke is also versioned, the phi node merging the return value of the promoted and original invoke instructions is placed in the "merge" block. Differential Revision: https://reviews.llvm.org/D40751 llvm-svn: 321210
* Fix more inconsistent line endings. NFC.Dimitry Andric2017-12-181-9/+9
| | | | llvm-svn: 321016
* [Memcpy Loop Lowering] Remove the fixed int8 lowering.Sean Fertile2017-12-181-80/+13
| | | | | | | | Switch over to the lowering that uses target supplied operand types. Differential Revision: https://reviews.llvm.org/D41201 llvm-svn: 320989
* [Memcpy Loop Lowering] Only calculate residual size/bytes copied when needed.Sean Fertile2017-12-161-6/+10
| | | | | | | | If the loop operand type is int8 then there will be no residual loop for the unknown size expansion. Dont create the residual-size and bytes-copied values when they are not needed. llvm-svn: 320929
* [SimplifyLibCalls] Inline calls to cabs when it's safe to do soHal Finkel2017-12-161-0/+33
| | | | | | | | | | | | When unsafe algerbra is allowed calls to cabs(r) can be replaced by: sqrt(creal(r)*creal(r) + cimag(r)*cimag(r)) Patch by Paul Walker, thanks! Differential Revision: https://reviews.llvm.org/D40069 llvm-svn: 320901
* [Memcpy Loop Lowering] Insert loop BB inbetween the split BB.Sean Fertile2017-12-151-2/+3
| | | | | | | | | | | The original memcpy expansion inserted the loop basic block inbetween the 2 new basic blocks created by splitting the original block the memcpy call was in. This commit makes the new memcpy expansion do the same to keep the layout of the IR matching between the old and new implementations. Differential Review: https://reviews.llvm.org/D41197 llvm-svn: 320848
* fix typo in comment and remove inaccurate comment; NFCSanjay Patel2017-12-151-2/+0
| | | | llvm-svn: 320838
* [SimplifyCFG] don't sink common insts too soon (PR34603)Sanjay Patel2017-12-141-1/+1
| | | | | | | | | | | | This should solve: https://bugs.llvm.org/show_bug.cgi?id=34603 ...by preventing SimplifyCFG from altering redundant instructions before early-cse has a chance to run. It changes the default (canonical-forming) behavior of SimplifyCFG, so we're only doing the sinking transform later in the optimization pipeline. Differential Revision: https://reviews.llvm.org/D38566 llvm-svn: 320749
* [LV] Support efficient vectorization of an induction with redundant castsDorit Nuzman2017-12-141-8/+133
| | | | | | | | | | | | | | | | | | | | | | | | | | | | D30041 extended SCEVPredicateRewriter to improve handling of Phi nodes whose update chain involves casts; PSCEV can now build an AddRecurrence for some forms of such phi nodes, under the proper runtime overflow test. This means that we can identify such phi nodes as an induction, and the loop-vectorizer can now vectorize such inductions, however inefficiently. The vectorizer doesn't know that it can ignore the casts, and so it vectorizes them. This patch records the casts in the InductionDescriptor, so that they could be marked to be ignored for cost calculation (we use VecValuesToIgnore for that) and ignored for vectorization/widening/scalarization (i.e. treated as TriviallyDead). In addition to marking all these casts to be ignored, we also need to make sure that each cast is mapped to the right vector value in the vector loop body (be it a widened, vectorized, or scalarized induction). So whenever an induction phi is mapped to a vector value (during vectorization/widening/ scalarization), we also map the respective cast instruction (if exists) to that vector value. (If the phi-update sequence of an induction involves more than one cast, then the above mapping to vector value is relevant only for the last cast of the sequence as we allow only the "last cast" to be used outside the induction update chain itself). This is the last step in addressing PR30654. llvm-svn: 320672
* Reverting [JumpThreading] Preservation of DT and LVI across the passBrian M. Rzycki2017-12-132-173/+50
| | | | | | | Stage 2 bootstrap failed: http://lab.llvm.org:8011/builders/clang-x86_64-linux-selfhost-modules-2/builds/14434 llvm-svn: 320641
* Remove redundant includes from lib/Transforms.Michael Zolotukhin2017-12-138-10/+0
| | | | llvm-svn: 320628
* [JumpThreading] Preservation of DT and LVI across the passBrian M. Rzycki2017-12-132-50/+173
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: See D37528 for a previous (non-deferred) version of this patch and its description. Preserves dominance in a deferred manner using a new class DeferredDominance. This reduces the performance impact of updating the DominatorTree at every edge insertion and deletion. A user may call DDT->flush() within JumpThreading for an up-to-date DT. This patch currently has one flush() at the end of runImpl() to ensure DT is preserved across the pass. LVI is also preserved to help subsequent passes such as CorrelatedValuePropagation. LVI is simpler to maintain and is done immediately (not deferred). The code to perfom the preversation was minimally altered and was simply marked as preserved for the PassManager to be informed. This extends the analysis available to JumpThreading for future enhancements. One example is loop boundary threading. Reviewers: dberlin, kuhar, sebpop Reviewed By: kuhar, sebpop Subscribers: hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D40146 llvm-svn: 320612
* Split IndirectBr critical edges before PGO gen/use passes.Hiroshi Yamauchi2017-12-121-6/+28
| | | | | | | | | | | | | | | | | | | | | | Summary: The PGO gen/use passes currently fail with an assert failure if there's a critical edge whose source is an IndirectBr instruction and that edge needs to be instrumented. To avoid this in certain cases, split IndirectBr critical edges in the PGO gen/use passes. This works for blocks with single indirectbr predecessors, but not for those with multiple indirectbr predecessors (splitting an IndirectBr critical edge isn't always possible.) Reviewers: davidxl, xur Reviewed By: davidxl Subscribers: efriedma, llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D40699 llvm-svn: 320511
* [SimplifyLibCalls] propagate FMF when folding pow(x, -1.0) callSanjay Patel2017-12-101-14/+11
| | | | | | | Follow-up for a bug that's similar to: https://bugs.llvm.org/show_bug.cgi?id=35601 llvm-svn: 320312
* [SimplifyLibCalls] propagate FMF when folding pow(x, 2.0) call (PR35601)Sanjay Patel2017-12-101-1/+6
| | | | | | | This should fix the larger problem with sqrt shown in: https://bugs.llvm.org/show_bug.cgi?id=35601 llvm-svn: 320310
* [InlineFunction] Set debug loc for call to forward varargs.Florian Hahn2017-12-091-0/+1
| | | | | | | | | | | | Reviewers: aprantl, dblaikie, rnk Reviewed By: rnk Subscribers: eraman, llvm-commits, JDevlieghere Differential Revision: https://reviews.llvm.org/D40432 llvm-svn: 320252
* Generalize llvm::replaceDbgDeclare and actually support the use-case thatAdrian Prantl2017-12-082-5/+8
| | | | | | is mentioned in the documentation (inserting a deref before the plus_uconst). llvm-svn: 320203
* [CodeExtractor] Add debug locations for new call and branch instrs.Florian Hahn2017-12-081-1/+24
| | | | | | | | | | | | | | | | | | | | | | | Summary: If a partially inlined function has debug info, we have to add debug locations to the call instruction calling the outlined function. We use the debug location of the first instruction in the outlined function, as the introduced call transfers control to this statement and there is no other equivalent line in the source code. We also use the same debug location for the branch instruction added to jump from artificial entry block for the outlined function, which just jumps to the first actual basic block of the outlined function. Reviewers: davide, aprantl, rriddle, dblaikie, danielcdh, wmi Reviewed By: aprantl, rriddle, danielcdh Subscribers: eraman, JDevlieghere, llvm-commits Differential Revision: https://reviews.llvm.org/D40413 llvm-svn: 320199
* [PGO] Make indirect call promotion a utilityMatthew Simpson2017-12-062-0/+329
| | | | | | | | | | | | | | | | This patch factors out the main code transformation utilities in the pgo-driven indirect call promotion pass and places them in Transforms/Utils. The change is intended to be a non-functional change, letting non-pgo-driven passes share a common implementation with the existing pgo-driven pass. The common utilities are used to conditionally promote indirect call sites to direct call sites. They perform the underlying transformation, and do not consider profile information. The pgo-specific details (e.g., the computation of branch weight metadata) have been left in the indirect call promotion pass. Differential Revision: https://reviews.llvm.org/D40658 llvm-svn: 319963
* [InlineFunction] Only replace call if there are VarArgs to forward.Florian Hahn2017-12-061-1/+2
| | | | | | | | | | | | | | | | Summary: There is no need to replace the original call instruction if no VarArgs need to be forwarded. Reviewers: davide, rnk, majnemer, efriedma Reviewed By: efriedma Subscribers: eraman, llvm-commits Differential Revision: https://reviews.llvm.org/D40412 llvm-svn: 319947
* [LoopUtils] simplify createTargetReduction(); NFCISanjay Patel2017-12-061-41/+25
| | | | llvm-svn: 319946
* [LoopUtils] fix variable name to match FMF vocabulary; NFCSanjay Patel2017-12-061-4/+4
| | | | llvm-svn: 319928
* Bail out of a SimplifyCFG switch table opt at undef values.Mikael Holmen2017-12-051-1/+1
| | | | | | | | | | | | | | | | | | | Summary: A true or false result is expected from a comparison, but it seems the possibility of undef was overlooked, which could lead to a failed assert. This is fixed by this patch by bailing out if we encounter undef. The bug is old and the assert has been there since the end of 2014, so it seems this is unusual enough to forego optimization. Patch by JesperAntonsson. Reviewers: spatel, eeckstein, hans Reviewed By: hans Subscribers: uabelho, llvm-commits Differential Revision: https://reviews.llvm.org/D40639 llvm-svn: 319768
* Move splitIndirectCriticalEdges() to BasicBlockUtils.h.Hiroshi Yamauchi2017-12-041-0/+139
| | | | | | | | | | | | | | | | Summary: Move splitIndirectCriticalEdges() from CodeGenPrepare to BasicBlockUtils.h so that it can be called from other places. Reviewers: davidxl Reviewed By: davidxl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D40750 llvm-svn: 319689
* [BypassSlowDivision] Improve our handling of divisions by constantsSanjoy Das2017-12-041-7/+13
| | | | | | | | | | | | | | | | | | | (This reapplies r314253. r314253 was reverted on r314482 because of a correctness regression on P100, but that regression was identified to be something else.) Summary: Don't bail out on constant divisors for divisions that can be narrowed without introducing control flow . This gives us a 32 bit multiply instead of an emulated 64 bit multiply in the generated PTX assembly. Reviewers: jlebar Subscribers: jholewinski, mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D38265 llvm-svn: 319677
* [IndVars] Fix a bug introduced in r317012Philip Reames2017-12-011-3/+13
| | | | | | | | Turns out we can have comparisons which are indirect users of the induction variable that we can make invariant. In this case, there is no loop invariant value contributing and we'd fail an assert. The test case was found by a java fuzzer and reduced. It's a real cornercase. You have to have a static loop which we've already proven only executes once, but haven't broken the backedge on, and an inner phi whose result can be constant folded by SCEV using exit count reasoning but not proven by isKnownPredicate. To my knowledge, only the fuzzer has hit this case. llvm-svn: 319583
* Revert r319537: Bail out of a SimplifyCFG switch table opt at undef values.Mikael Holmen2017-12-011-1/+1
| | | | | | Broke build bots so reverting. llvm-svn: 319539
* Bail out of a SimplifyCFG switch table opt at undef values.Mikael Holmen2017-12-011-1/+1
| | | | | | | | | | | | | | | | | | | Summary: A true or false result is expected from a comparison, but it seems the possibility of undef was overlooked, which could lead to a failed assert. This is fixed by this patch by bailing out if we encounter undef. The bug is old and the assert has been there since the end of 2014, so it seems this is unusual enough to forego optimization. Patch by: JesperAntonsson Reviewers: spatel, eeckstein, hans Reviewed By: hans Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D40639 llvm-svn: 319537
* Mark all library options as hidden.Zachary Turner2017-12-012-4/+6
| | | | | | | | | | | | | | | | | These command line options are not intended for public use, and often don't even make sense in the context of a particular tool anyway. About 90% of them are already hidden, but when people add new options they forget to hide them, so if you were to make a brand new tool today, link against one of LLVM's libraries, and run tool -help you would get a bunch of junk that doesn't make sense for the tool you're writing. This patch hides these options. The real solution is to not have libraries defining command line options, but that's a much larger effort and not something I'm prepared to take on. Differential Revision: https://reviews.llvm.org/D40674 llvm-svn: 319505
* EntryExitInstrumenter: set DebugLocs on the inserted call instructions (PR35412)Hans Wennborg2017-11-281-5/+20
| | | | | | | Apparently the verifier requires that inlineable calls in a function with debug info have debug locations. llvm-svn: 319199
* This reverts commit r319096 and r319097.Rafael Espindola2017-11-282-162/+26
| | | | | | | | | Revert "[SROA] Propagate !range metadata when moving loads." Revert "[Mem2Reg] Clang-format unformatted parts of this file. NFCI." Davide says they broke a bot. llvm-svn: 319131
* [Mem2Reg] Clang-format unformatted parts of this file. NFCI.Davide Italiano2017-11-271-28/+23
| | | | llvm-svn: 319097
* [SROA] Propagate !range metadata when moving loads.Davide Italiano2017-11-272-24/+165
| | | | | | | | | | | | | This tries to propagate !range metadata to a pre-existing load when a load is optimized out. This is done instead of adding an assume because converting loads to and from assumes creates a lot of IR. Patch by Ariel Ben-Yehuda. Differential Revision: https://reviews.llvm.org/D37216 llvm-svn: 319096
* Inliner: Don't mark notail calls with the 'tail' attributeArnold Schwaighofer2017-11-271-1/+2
| | | | | | | | | | | | enum TailCallKind { TCK_None = 0, TCK_Tail = 1, TCK_MustTail = 2, TCK_NoTail = 3 }; TCK_NoTail is greater than TCK_Tail so taking the min does not do the correct thing. rdar://35639547 llvm-svn: 319075
* EntryExitInstrumenter: support __cyg_profile_func_enter_bareHans Wennborg2017-11-211-1/+2
| | | | | | It works just like __cyg_profile_func_enter but takes no arguments. llvm-svn: 318783
* [LibCallSimplifier] allow splat vectors for pow(x, 0.5) -> sqrt() transformsSanjay Patel2017-11-191-8/+7
| | | | llvm-svn: 318629
* [LibCallSimplifier] partly fix pow(x, 0.5) -> sqrt() transformsSanjay Patel2017-11-191-32/+49
| | | | | | | | | | | | | | As the first test shows, we could transform an llvm intrinsic which never sets errno into a libcall which could set errno (even though it's marked readnone?), so that's not ideal. It's possible that we can also transform a libcall which could set errno to an intrinsic given the fast-math-flags constraint, but that's deferred to determine exactly which set of FMF are needed. Differential Revision: https://reviews.llvm.org/D40150 llvm-svn: 318628
* [asan] Add a full redzone after every stack variableWalter Lee2017-11-181-3/+6
| | | | | | | | | We were not doing that for large shadow granularity. Also add more stack frame layout tests for large shadow granularity. Differential Revision: https://reviews.llvm.org/D39475 llvm-svn: 318581
OpenPOWER on IntegriCloud