summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms/Utils
Commit message (Collapse)AuthorAgeFilesLines
* [LV] Support efficient vectorization of an induction with redundant castsDorit Nuzman2017-12-141-8/+133
| | | | | | | | | | | | | | | | | | | | | | | | | | | | D30041 extended SCEVPredicateRewriter to improve handling of Phi nodes whose update chain involves casts; PSCEV can now build an AddRecurrence for some forms of such phi nodes, under the proper runtime overflow test. This means that we can identify such phi nodes as an induction, and the loop-vectorizer can now vectorize such inductions, however inefficiently. The vectorizer doesn't know that it can ignore the casts, and so it vectorizes them. This patch records the casts in the InductionDescriptor, so that they could be marked to be ignored for cost calculation (we use VecValuesToIgnore for that) and ignored for vectorization/widening/scalarization (i.e. treated as TriviallyDead). In addition to marking all these casts to be ignored, we also need to make sure that each cast is mapped to the right vector value in the vector loop body (be it a widened, vectorized, or scalarized induction). So whenever an induction phi is mapped to a vector value (during vectorization/widening/ scalarization), we also map the respective cast instruction (if exists) to that vector value. (If the phi-update sequence of an induction involves more than one cast, then the above mapping to vector value is relevant only for the last cast of the sequence as we allow only the "last cast" to be used outside the induction update chain itself). This is the last step in addressing PR30654. llvm-svn: 320672
* Reverting [JumpThreading] Preservation of DT and LVI across the passBrian M. Rzycki2017-12-132-173/+50
| | | | | | | Stage 2 bootstrap failed: http://lab.llvm.org:8011/builders/clang-x86_64-linux-selfhost-modules-2/builds/14434 llvm-svn: 320641
* Remove redundant includes from lib/Transforms.Michael Zolotukhin2017-12-138-10/+0
| | | | llvm-svn: 320628
* [JumpThreading] Preservation of DT and LVI across the passBrian M. Rzycki2017-12-132-50/+173
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: See D37528 for a previous (non-deferred) version of this patch and its description. Preserves dominance in a deferred manner using a new class DeferredDominance. This reduces the performance impact of updating the DominatorTree at every edge insertion and deletion. A user may call DDT->flush() within JumpThreading for an up-to-date DT. This patch currently has one flush() at the end of runImpl() to ensure DT is preserved across the pass. LVI is also preserved to help subsequent passes such as CorrelatedValuePropagation. LVI is simpler to maintain and is done immediately (not deferred). The code to perfom the preversation was minimally altered and was simply marked as preserved for the PassManager to be informed. This extends the analysis available to JumpThreading for future enhancements. One example is loop boundary threading. Reviewers: dberlin, kuhar, sebpop Reviewed By: kuhar, sebpop Subscribers: hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D40146 llvm-svn: 320612
* Split IndirectBr critical edges before PGO gen/use passes.Hiroshi Yamauchi2017-12-121-6/+28
| | | | | | | | | | | | | | | | | | | | | | Summary: The PGO gen/use passes currently fail with an assert failure if there's a critical edge whose source is an IndirectBr instruction and that edge needs to be instrumented. To avoid this in certain cases, split IndirectBr critical edges in the PGO gen/use passes. This works for blocks with single indirectbr predecessors, but not for those with multiple indirectbr predecessors (splitting an IndirectBr critical edge isn't always possible.) Reviewers: davidxl, xur Reviewed By: davidxl Subscribers: efriedma, llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D40699 llvm-svn: 320511
* [SimplifyLibCalls] propagate FMF when folding pow(x, -1.0) callSanjay Patel2017-12-101-14/+11
| | | | | | | Follow-up for a bug that's similar to: https://bugs.llvm.org/show_bug.cgi?id=35601 llvm-svn: 320312
* [SimplifyLibCalls] propagate FMF when folding pow(x, 2.0) call (PR35601)Sanjay Patel2017-12-101-1/+6
| | | | | | | This should fix the larger problem with sqrt shown in: https://bugs.llvm.org/show_bug.cgi?id=35601 llvm-svn: 320310
* [InlineFunction] Set debug loc for call to forward varargs.Florian Hahn2017-12-091-0/+1
| | | | | | | | | | | | Reviewers: aprantl, dblaikie, rnk Reviewed By: rnk Subscribers: eraman, llvm-commits, JDevlieghere Differential Revision: https://reviews.llvm.org/D40432 llvm-svn: 320252
* Generalize llvm::replaceDbgDeclare and actually support the use-case thatAdrian Prantl2017-12-082-5/+8
| | | | | | is mentioned in the documentation (inserting a deref before the plus_uconst). llvm-svn: 320203
* [CodeExtractor] Add debug locations for new call and branch instrs.Florian Hahn2017-12-081-1/+24
| | | | | | | | | | | | | | | | | | | | | | | Summary: If a partially inlined function has debug info, we have to add debug locations to the call instruction calling the outlined function. We use the debug location of the first instruction in the outlined function, as the introduced call transfers control to this statement and there is no other equivalent line in the source code. We also use the same debug location for the branch instruction added to jump from artificial entry block for the outlined function, which just jumps to the first actual basic block of the outlined function. Reviewers: davide, aprantl, rriddle, dblaikie, danielcdh, wmi Reviewed By: aprantl, rriddle, danielcdh Subscribers: eraman, JDevlieghere, llvm-commits Differential Revision: https://reviews.llvm.org/D40413 llvm-svn: 320199
* [PGO] Make indirect call promotion a utilityMatthew Simpson2017-12-062-0/+329
| | | | | | | | | | | | | | | | This patch factors out the main code transformation utilities in the pgo-driven indirect call promotion pass and places them in Transforms/Utils. The change is intended to be a non-functional change, letting non-pgo-driven passes share a common implementation with the existing pgo-driven pass. The common utilities are used to conditionally promote indirect call sites to direct call sites. They perform the underlying transformation, and do not consider profile information. The pgo-specific details (e.g., the computation of branch weight metadata) have been left in the indirect call promotion pass. Differential Revision: https://reviews.llvm.org/D40658 llvm-svn: 319963
* [InlineFunction] Only replace call if there are VarArgs to forward.Florian Hahn2017-12-061-1/+2
| | | | | | | | | | | | | | | | Summary: There is no need to replace the original call instruction if no VarArgs need to be forwarded. Reviewers: davide, rnk, majnemer, efriedma Reviewed By: efriedma Subscribers: eraman, llvm-commits Differential Revision: https://reviews.llvm.org/D40412 llvm-svn: 319947
* [LoopUtils] simplify createTargetReduction(); NFCISanjay Patel2017-12-061-41/+25
| | | | llvm-svn: 319946
* [LoopUtils] fix variable name to match FMF vocabulary; NFCSanjay Patel2017-12-061-4/+4
| | | | llvm-svn: 319928
* Bail out of a SimplifyCFG switch table opt at undef values.Mikael Holmen2017-12-051-1/+1
| | | | | | | | | | | | | | | | | | | Summary: A true or false result is expected from a comparison, but it seems the possibility of undef was overlooked, which could lead to a failed assert. This is fixed by this patch by bailing out if we encounter undef. The bug is old and the assert has been there since the end of 2014, so it seems this is unusual enough to forego optimization. Patch by JesperAntonsson. Reviewers: spatel, eeckstein, hans Reviewed By: hans Subscribers: uabelho, llvm-commits Differential Revision: https://reviews.llvm.org/D40639 llvm-svn: 319768
* Move splitIndirectCriticalEdges() to BasicBlockUtils.h.Hiroshi Yamauchi2017-12-041-0/+139
| | | | | | | | | | | | | | | | Summary: Move splitIndirectCriticalEdges() from CodeGenPrepare to BasicBlockUtils.h so that it can be called from other places. Reviewers: davidxl Reviewed By: davidxl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D40750 llvm-svn: 319689
* [BypassSlowDivision] Improve our handling of divisions by constantsSanjoy Das2017-12-041-7/+13
| | | | | | | | | | | | | | | | | | | (This reapplies r314253. r314253 was reverted on r314482 because of a correctness regression on P100, but that regression was identified to be something else.) Summary: Don't bail out on constant divisors for divisions that can be narrowed without introducing control flow . This gives us a 32 bit multiply instead of an emulated 64 bit multiply in the generated PTX assembly. Reviewers: jlebar Subscribers: jholewinski, mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D38265 llvm-svn: 319677
* [IndVars] Fix a bug introduced in r317012Philip Reames2017-12-011-3/+13
| | | | | | | | Turns out we can have comparisons which are indirect users of the induction variable that we can make invariant. In this case, there is no loop invariant value contributing and we'd fail an assert. The test case was found by a java fuzzer and reduced. It's a real cornercase. You have to have a static loop which we've already proven only executes once, but haven't broken the backedge on, and an inner phi whose result can be constant folded by SCEV using exit count reasoning but not proven by isKnownPredicate. To my knowledge, only the fuzzer has hit this case. llvm-svn: 319583
* Revert r319537: Bail out of a SimplifyCFG switch table opt at undef values.Mikael Holmen2017-12-011-1/+1
| | | | | | Broke build bots so reverting. llvm-svn: 319539
* Bail out of a SimplifyCFG switch table opt at undef values.Mikael Holmen2017-12-011-1/+1
| | | | | | | | | | | | | | | | | | | Summary: A true or false result is expected from a comparison, but it seems the possibility of undef was overlooked, which could lead to a failed assert. This is fixed by this patch by bailing out if we encounter undef. The bug is old and the assert has been there since the end of 2014, so it seems this is unusual enough to forego optimization. Patch by: JesperAntonsson Reviewers: spatel, eeckstein, hans Reviewed By: hans Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D40639 llvm-svn: 319537
* Mark all library options as hidden.Zachary Turner2017-12-012-4/+6
| | | | | | | | | | | | | | | | | These command line options are not intended for public use, and often don't even make sense in the context of a particular tool anyway. About 90% of them are already hidden, but when people add new options they forget to hide them, so if you were to make a brand new tool today, link against one of LLVM's libraries, and run tool -help you would get a bunch of junk that doesn't make sense for the tool you're writing. This patch hides these options. The real solution is to not have libraries defining command line options, but that's a much larger effort and not something I'm prepared to take on. Differential Revision: https://reviews.llvm.org/D40674 llvm-svn: 319505
* EntryExitInstrumenter: set DebugLocs on the inserted call instructions (PR35412)Hans Wennborg2017-11-281-5/+20
| | | | | | | Apparently the verifier requires that inlineable calls in a function with debug info have debug locations. llvm-svn: 319199
* This reverts commit r319096 and r319097.Rafael Espindola2017-11-282-162/+26
| | | | | | | | | Revert "[SROA] Propagate !range metadata when moving loads." Revert "[Mem2Reg] Clang-format unformatted parts of this file. NFCI." Davide says they broke a bot. llvm-svn: 319131
* [Mem2Reg] Clang-format unformatted parts of this file. NFCI.Davide Italiano2017-11-271-28/+23
| | | | llvm-svn: 319097
* [SROA] Propagate !range metadata when moving loads.Davide Italiano2017-11-272-24/+165
| | | | | | | | | | | | | This tries to propagate !range metadata to a pre-existing load when a load is optimized out. This is done instead of adding an assume because converting loads to and from assumes creates a lot of IR. Patch by Ariel Ben-Yehuda. Differential Revision: https://reviews.llvm.org/D37216 llvm-svn: 319096
* Inliner: Don't mark notail calls with the 'tail' attributeArnold Schwaighofer2017-11-271-1/+2
| | | | | | | | | | | | enum TailCallKind { TCK_None = 0, TCK_Tail = 1, TCK_MustTail = 2, TCK_NoTail = 3 }; TCK_NoTail is greater than TCK_Tail so taking the min does not do the correct thing. rdar://35639547 llvm-svn: 319075
* EntryExitInstrumenter: support __cyg_profile_func_enter_bareHans Wennborg2017-11-211-1/+2
| | | | | | It works just like __cyg_profile_func_enter but takes no arguments. llvm-svn: 318783
* [LibCallSimplifier] allow splat vectors for pow(x, 0.5) -> sqrt() transformsSanjay Patel2017-11-191-8/+7
| | | | llvm-svn: 318629
* [LibCallSimplifier] partly fix pow(x, 0.5) -> sqrt() transformsSanjay Patel2017-11-191-32/+49
| | | | | | | | | | | | | | As the first test shows, we could transform an llvm intrinsic which never sets errno into a libcall which could set errno (even though it's marked readnone?), so that's not ideal. It's possible that we can also transform a libcall which could set errno to an intrinsic given the fast-math-flags constraint, but that's deferred to determine exactly which set of FMF are needed. Differential Revision: https://reviews.llvm.org/D40150 llvm-svn: 318628
* [asan] Add a full redzone after every stack variableWalter Lee2017-11-181-3/+6
| | | | | | | | | We were not doing that for large shadow granularity. Also add more stack frame layout tests for large shadow granularity. Differential Revision: https://reviews.llvm.org/D39475 llvm-svn: 318581
* [PredicateInfo] Add comment about why we require stable sortMandeep Singh Grang2017-11-171-0/+5
| | | | llvm-svn: 318487
* [EntryExitInstrumenter] Placate GCC, the semicolon is redundant. NFCI.Davide Italiano2017-11-141-3/+2
| | | | llvm-svn: 318217
* Make salvageDebugInfo of casts work for dbg.declare and dbg.addrReid Kleckner2017-11-141-6/+16
| | | | | | | | | | | | | | | | | | | | | Summary: Instcombine (and probably other passes) sometimes want to change the type of an alloca. To do this, they generally create a new alloca with the desired type, create a bitcast to make the new pointer type match the old pointer type, replace all uses with the cast, and then simplify the casts. We already knew how to salvage dbg.value instructions when removing casts, but we can extend it to cover dbg.addr and dbg.declare. Fixes a debug info quality issue uncovered in Chromium in http://crbug.com/784609 Reviewers: aprantl, vsk Subscribers: hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D40042 llvm-svn: 318203
* Rename CountingFunctionInserter and use for both mcount and cygprofile ↵Hans Wennborg2017-11-142-0/+149
| | | | | | | | | | | | | | | | | | | | | | calls, before and after inlining Clang implements the -finstrument-functions flag inherited from GCC, which inserts calls to __cyg_profile_func_{enter,exit} on function entry and exit. This is useful for getting a trace of how the functions in a program are executed. Normally, the calls remain even if a function is inlined into another function, but it is useful to be able to turn this off for users who are interested in a lower-level trace, i.e. one that reflects what functions are called post-inlining. (We use this to generate link order files for Chromium.) LLVM already has a pass for inserting similar instrumentation calls to mcount(), which it does after inlining. This patch renames and extends that pass to handle calls both to mcount and the cygprofile functions, before and/or after inlining as controlled by function attributes. Differential Revision: https://reviews.llvm.org/D39287 llvm-svn: 318195
* [PredicateInfo] Stable sort ValueDFS to remove non-deterministic orderingMandeep Singh Grang2017-11-141-1/+1
| | | | | | | | | | | | | | Summary: This fixes failure in Transforms/Util/PredicateInfo/testandor.ll uncovered by D39245. Reviewers: dberlin Reviewed By: dberlin Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D39630 llvm-svn: 318165
* [CodeExtractor] Add missing AllowVarArgs initialization.Florian Hahn2017-11-131-2/+3
| | | | llvm-svn: 318029
* [PartialInliner] Inline vararg functions that forward varargs.Florian Hahn2017-11-132-21/+64
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This patch extends the partial inliner to support inlining parts of vararg functions, if the vararg handling is done in the outlined part. It adds a `ForwardVarArgsTo` argument to InlineFunction. If it is non-null, all varargs passed to the inlined function will be added to all calls to `ForwardVarArgsTo`. The partial inliner takes care to only pass `ForwardVarArgsTo` if the varargs handing is done in the outlined function. It checks that vastart is not part of the function to be inlined. `test/Transforms/CodeExtractor/PartialInlineNoInline.ll` (already part of the repo) checks we do not do partial inlining if vastart is used in a basic block that will be inlined. Reviewers: davide, davidxl, grosser Reviewed By: davide, davidxl, grosser Subscribers: gyiu, grosser, eraman, llvm-commits Differential Revision: https://reviews.llvm.org/D39607 llvm-svn: 318028
* [SimplifyCFG] Use auto * when the type is obvious. NFCI.Davide Italiano2017-11-101-11/+8
| | | | llvm-svn: 317923
* Add a wrapper function to set branch weights metadata.Easwaran Raman2017-11-091-28/+34
| | | | | | | | | | | | | | Summary: This wrapper checks if there is at least one non-zero weight before setting the metadata. Reviewers: davidxl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D39872 llvm-svn: 317845
* Add an @llvm.sideeffect intrinsicDan Gohman2017-11-081-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | This patch implements Chandler's idea [0] for supporting languages that require support for infinite loops with side effects, such as Rust, providing part of a solution to bug 965 [1]. Specifically, it adds an `llvm.sideeffect()` intrinsic, which has no actual effect, but which appears to optimization passes to have obscure side effects, such that they don't optimize away loops containing it. It also teaches several optimization passes to ignore this intrinsic, so that it doesn't significantly impact optimization in most cases. As discussed on llvm-dev [2], this patch is the first of two major parts. The second part, to change LLVM's semantics to have defined behavior on infinite loops by default, with a function attribute for opting into potential-undefined-behavior, will be implemented and posted for review in a separate patch. [0] http://lists.llvm.org/pipermail/llvm-dev/2015-July/088103.html [1] https://bugs.llvm.org/show_bug.cgi?id=965 [2] http://lists.llvm.org/pipermail/llvm-dev/2017-October/118632.html Differential Revision: https://reviews.llvm.org/D38336 llvm-svn: 317729
* InstCombine: salvage the debug info of DCE'ed add instructions.Adrian Prantl2017-11-061-12/+23
| | | | | | rdar://problem/31209283 llvm-svn: 317522
* [IR] redefine 'UnsafeAlgebra' / 'reassoc' fast-math-flags and add 'trans' ↵Sanjay Patel2017-11-062-25/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | fast-math-flag As discussed on llvm-dev: http://lists.llvm.org/pipermail/llvm-dev/2016-November/107104.html and again more recently: http://lists.llvm.org/pipermail/llvm-dev/2017-October/118118.html ...this is a step in cleaning up our fast-math-flags implementation in IR to better match the capabilities of both clang's user-visible flags and the backend's flags for SDNode. As proposed in the above threads, we're replacing the 'UnsafeAlgebra' bit (which had the 'umbrella' meaning that all flags are set) with a new bit that only applies to algebraic reassociation - 'AllowReassoc'. We're also adding a bit to allow approximations for library functions called 'ApproxFunc' (this was initially proposed as 'libm' or similar). ...and we're out of bits. 7 bits ought to be enough for anyone, right? :) FWIW, I did look at getting this out of SubclassOptionalData via SubclassData (spacious 16-bits), but that's apparently already used for other purposes. Also, I don't think we can just add a field to FPMathOperator because Operator is not intended to be instantiated. We'll defer movement of FMF to another day. We keep the 'fast' keyword. I thought about removing that, but seeing IR like this: %f.fast = fadd reassoc nnan ninf nsz arcp contract afn float %op1, %op2 ...made me think we want to keep the shortcut synonym. Finally, this change is binary incompatible with existing IR as seen in the compatibility tests. This statement: "Newer releases can ignore features from older releases, but they cannot miscompile them. For example, if nsw is ever replaced with something else, dropping it would be a valid way to upgrade the IR." ( http://llvm.org/docs/DeveloperPolicy.html#ir-backwards-compatibility ) ...provides the flexibility we want to make this change without requiring a new IR version. Ie, we're not loosening the FP strictness of existing IR. At worst, we will fail to optimize some previously 'fast' code because it's no longer recognized as 'fast'. This should get fixed as we audit/squash all of the uses of 'isFast()'. Note: an inter-dependent clang commit to use the new API name should closely follow commit. Differential Revision: https://reviews.llvm.org/D39304 llvm-svn: 317488
* [LTO][ThinLTO] Use the linker resolutions to mark global values as dso_local.Sean Fertile2017-11-041-0/+17
| | | | | | | | | | | | | Now that we have a way to mark GlobalValues as local we can use the symbol resolutions that the linker plugin provides as part of lto/thinlto link step to refine the compilers view on what symbols will end up being local. Originally commited as r317374, but reverted in r317395 to update some missed tests. Differential Revision: https://reviews.llvm.org/D35702 llvm-svn: 317408
* Revert "[LTO][ThinLTO] Use the linker resolutions to mark global values ..."Sean Fertile2017-11-041-17/+0
| | | | | | | | | Changes more tests then expected on one of the build bots. reverting to investigate. This reverts https://llvm.org/svn/llvm-project/llvm/trunk@317374 llvm-svn: 317395
* Invoke salvageDebugInfo from CodeGenPrepare's SinkCast()Adrian Prantl2017-11-031-1/+1
| | | | | | | | | | This preserves the debug info for the cast operation in the original location. rdar://problem/33460652 Reapplied r317340 with the test moved into an ARM-specific directory. llvm-svn: 317375
* [LTO][ThinLTO] Use the linker resolutions to mark global values as dso_local.Sean Fertile2017-11-031-0/+17
| | | | | | | | | | Now that we have a way to mark GlobalValues as local we can use the symbol resolutions that the linker plugin provides as part of lto/thinlto link step to refine the compilers view on what symbols will end up being local. Differential Revision: https://reviews.llvm.org/D35702 llvm-svn: 317374
* [SimplifyCFG] When merging conditional stores, don't count the store we're ↵Craig Topper2017-11-031-1/+3
| | | | | | | | | | | | merging against the PHINodeFoldingThreshold Merging conditional stores tries to check to see if the code is if convertible after the store is moved. But the store hasn't been moved yet so its being counted against the threshold. The patch adds 1 to the threshold comparison to make sure we don't count the store. I've adjusted a test to use a lower threshold to ensure we still do that conversion with the lower threshold. Differential Revision: https://reviews.llvm.org/D39570 llvm-svn: 317368
* Add llvm::for_each as a range-based extensions to <algorithm> and make use ↵Aaron Ballman2017-11-031-9/+9
| | | | | | of it in some cases where it is a more clear alternative to std::for_each. llvm-svn: 317356
* Revert "Invoke salvageDebugInfo from CodeGenPrepare's SinkCast()"Adrian Prantl2017-11-031-1/+1
| | | | | | This reverts commit 317342 while investigating bot breakage. llvm-svn: 317345
* Invoke salvageDebugInfo from CodeGenPrepare's SinkCast()Adrian Prantl2017-11-031-1/+1
| | | | | | | | This preserves the debug info for the cast operation in the original location. rdar://problem/33460652 llvm-svn: 317340
OpenPOWER on IntegriCloud