summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
* Fix mapping of unmaterialized global values during metadata linkingTeresa Johnson2015-11-151-5/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The patch to move metadata linking after global value linking didn't correctly map unmaterialized global values to null as desired. They were in fact mapped to the source copy. It largely worked by accident since most module linker clients destroyed the source module which caused the source GVs to be replaced by null, but caused a failure with LTO linking on Windows: http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20151109/312869.html The problem is that a null return value from materializeValueFor is handled by mapping the value to self. This is the desired behavior when materializeValueFor is passed a non-GlobalValue. The problem is how to distinguish that case from the case where we really do want to map to null. This patch addresses this by passing in a new flag to the value mapper indicating that unmapped global values should be mapped to null. Other Value types are handled as before. Note that the documented behavior of asserting on unmapped values when the flag RF_IgnoreMissingValues isn't set is currently disabled with FIXME notes due to bootstrap failures. I modified these disabled asserts so when they are eventually enabled again it won't assert for the unmapped values when the new RF_NullMapMissingGlobalValues flag is set. I also considered using a callback into the value materializer, but a flag seemed cleaner given that there are already existing flags. I also considered modifying materializeValueFor to return the input value when we want to map to source and then treat a null return to mean map to null. However, there are other value materializer subclasses that implement materializeValueFor, and they would all need to be audited and the return values possibly changed, which seemed error-prone. Reviewers: dexonsmith, joker.eph Subscribers: pcc, llvm-commits Differential Revision: http://reviews.llvm.org/D14682 llvm-svn: 253170
* [GlobalOpt] Demote globals to locals more aggressivelyJames Molloy2015-11-151-7/+76
| | | | | | | | | | | | | | | | Global to local demotion can speed up programs that use globals a lot. It is particularly useful with LTO, when the entire call graph is known and most functions have been internalized. For a global to be demoted, it must only be accessed by one function and that function: 1. Must never recurse directly or indirectly, else the GV would be clobbered. 2. Must never rely on the value in GV at the start of the function (apart from the initializer). GlobalOpt can already do this, but it is hamstrung and only ever tries to demote globals inside "main", because C++ gives extra guarantees about how main is called - once and only once. In LTO mode, we can often prove the first property (if the function is internal by this point, we know enough about the callgraph to determine if it could possibly recurse). FunctionAttrs now infers the "norecurse" attribute for this reason. The second property can be proven for a subset of functions by proving that all loads from GV are dominated by a store to GV. This is conservative in the name of compile time - this only requires a DominatorTree which is fairly cheap in the grand scheme of things. We could do more fancy stuff with MemoryDependenceAnalysis too to catch more cases but this appears to catch most of the useful ones in my testing. llvm-svn: 253168
* Fixed GEP visitor in the InstCombine pass.Elena Demikhovsky2015-11-151-5/+10
| | | | | | | | | | | | | The current implementation of GEP visitor in InstCombine fails with assertion on Vector GEP with mix of scalar and vector types, like this: getelementptr double, double* %a, <8 x i32> %i (It fails to create a "sext" from <8 x i32> to <8 x i64>) I fixed it and added some tests. Differential Revision: http://reviews.llvm.org/D14485 llvm-svn: 253162
* Don't recompute LCSSA after loop-unrolling when possible.Michael Zolotukhin2015-11-141-1/+11
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: Currently we always recompute LCSSA for outer loops after unrolling an inner loop. That leads to compile time problem when we have big loop nests, and we can solve it by avoiding unnecessary work. For instance, if w eonly do partial unrolling, we don't break LCSSA, so we don't need to rebuild it. Also, if all exits from the inner loop are inside the enclosing loop, then complete unrolling won't break LCSSA either. I replaced unconditional LCSSA recomputation with conditional recomputation + unconditional assert and added several tests, which were failing when I experimented with it. Soon I plan to follow up with a similar patch for recalculation of dominators tree. Reviewers: hfinkel, dexonsmith, bogner, joker.eph, chandlerc Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D14526 llvm-svn: 253126
* [LIR] Add support for creating memcpys from loops with a negative stride.Chad Rosier2015-11-131-12/+21
| | | | | | | | | | | | | | | This allows us to transform the below loop into a memcpy. void test(unsigned *__restrict__ a, unsigned *__restrict__ b) { for (int i = 2047; i >= 0; --i) { a[i] = b[i]; } } This is the memcpy version of r251518, which added support for memset with negative strided loops. llvm-svn: 253091
* [safestack] Rewrite isAllocaSafe using SCEV.Evgeniy Stepanov2015-11-131-108/+166
| | | | | | | | | | | | | | | Use ScalarEvolution to calculate memory access bounds. Handle function calls based on readnone/nocapture attributes. Handle memory intrinsics with constant size. This change improves both recall and precision of IsAllocaSafe. See the new tests (ex. BitCastWide) for the kind of code that was wrongly classified as safe. SCEV efficiency seems to be limited by the fact the SafeStack runs late (in CodeGenPrepare), and many loops are unrolled or otherwise not in LCSSA. llvm-svn: 253083
* Add a comment that should have made my last commit.Chad Rosier2015-11-131-0/+1
| | | | llvm-svn: 253063
* [LIR] Factor out the code to compute base ptr for negative strided loops.Chad Rosier2015-11-131-10/+15
| | | | | | This will allow for the code to be reused in the memcpy optimization. llvm-svn: 253061
* [GlobalOpt] Make sure all debug lines end with '\n'James Molloy2015-11-131-2/+2
| | | | | | GlobalVariable::print() used to emit a newline. It hasn't for a while now, but these debug lines weren't updated. llvm-svn: 253030
* [GlobalOpt] Coding style - remove function names from doxygen commentsJames Molloy2015-11-131-126/+115
| | | | | | Suggested by Mehdi in the review of D14148. llvm-svn: 253029
* Revert r252990.Akira Hatanaka2015-11-131-1/+34
| | | | | | Some of the buildbots are still failing. llvm-svn: 252999
* Provide a way to specify inliner's attribute compatibility and merging.Akira Hatanaka2015-11-131-34/+1
| | | | | | | | | | | | | | | | | | This reapplies r252949. I've changed the type of FuncName to be std::string instead of StringRef in emitFnAttrCompatCheck. Original commit message for r252949: Provide a way to specify inliner's attribute compatibility and merging rules using table-gen. NFC. This commit adds new classes CompatRule and MergeRule to Attributes.td, which are used to generate code to check attribute compatibility and merge attributes of the caller and callee. rdar://problem/19836465 llvm-svn: 252990
* [SimplifyLibCalls] Make a function shorter. NFC.Davide Italiano2015-11-121-10/+2
| | | | llvm-svn: 252970
* Revert r252949.Akira Hatanaka2015-11-121-1/+34
| | | | | | It broke some of the bots including clang-x64-ninja-win7. llvm-svn: 252951
* Provide a way to specify inliner's attribute compatibility and mergingAkira Hatanaka2015-11-121-34/+1
| | | | | | | | | | | | rules using table-gen. NFC. This commit adds new classes CompatRule and MergeRule to Attributes.td, which are used to generate code to check attribute compatibility and merge attributes of the caller and callee. rdar://problem/19836465 llvm-svn: 252949
* Revert "Fix bug 25440: GVN assertion after coercing loads"Tobias Grosser2015-11-121-41/+13
| | | | | | This reverts 252919 which broke LNT: MultiSource/Applications/SPASS llvm-svn: 252936
* [LIR] Minor refactoring. NFCI.Chad Rosier2015-11-121-21/+44
| | | | | | | This change prevents uninteresting stores from being inserted into the list of candidate stores for memset/memcpy conversion. llvm-svn: 252926
* Fix bug 25440: GVN assertion after coercing loadsWeiming Zhao2015-11-121-13/+41
| | | | | | | | | | | | | | | | Summary: when coercing loads, it inserts some instructions, which have no GV assigned. https://llvm.org/bugs/show_bug.cgi?id=25440 Reviewers: hfinkel, dberlin Subscribers: dberlin, llvm-commits Differential Revision: http://reviews.llvm.org/D14479 llvm-svn: 252919
* [InstCombine] Add trivial folding (bitreverse (bitreverse x)) -> xJames Molloy2015-11-121-0/+10
| | | | | | There are plenty more instcombines we could probably do with bitreverse, but this seems like a very obvious and trivial starting point and was brought up by Hal in his review. llvm-svn: 252879
* Revert "Revert "[FunctionAttrs] Identify norecurse functions""James Molloy2015-11-121-1/+78
| | | | | | This reapplies this patch, with test fixes. llvm-svn: 252871
* Revert "[FunctionAttrs] Identify norecurse functions"James Molloy2015-11-121-78/+1
| | | | | | This reverts commit r252862. This introduced test failures and I'm reverting while I investigate how this happened. llvm-svn: 252863
* [FunctionAttrs] Identify norecurse functionsJames Molloy2015-11-121-1/+78
| | | | | | | | | | | | | A function can be marked as norecurse if: * The SCC to which it belongs has cardinality 1; and either a) It does not call any non-norecurse function. This includes self-recursion; or b) It only has one callsite and the function that callsite is within is marked norecurse. a) is best propagated bottom-up and b) is best propagated top-down. We build up the norecurse attributes bottom-up using the existing SCC pass, and mark functions with no obvious recursion (but not provably norecurse) to sweep later, top-down. llvm-svn: 252862
* [LIR] General refactor to improve compile-time and simplify code.Chad Rosier2015-11-111-16/+26
| | | | | | | | | | | First create a list of candidates, then transform. This simplifies the code in that you have don't have to worry that you may be using an invalidated iterator. Previously, each time we created a memset/memcpy we would reevaluate the entire loop potentially resulting in lots of redundant work for large basic blocks. llvm-svn: 252817
* [IR] Add support for empty tokensDavid Majnemer2015-11-111-1/+3
| | | | | | | | | | | | | | When working with tokens, it is often the case that one has instructions which consume a token and produce a new token. Currently, we have no mechanism to represent an initial token state. Instead, we can create a notional "empty token" by inventing a new constant which captures the semantics we would like. This new constant is called ConstantTokenNone and is written textually as "token none". Differential Revision: http://reviews.llvm.org/D14581 llvm-svn: 252811
* SamplePGO - Fix PR 25482 - Do not rely on llvm.dbg.cu for discriminatorsDiego Novillo2015-11-111-2/+2
| | | | | | | | | | | | | | | The discriminators pass relied on the presence of llvm.dbg.cu to decide whether to add discriminators, but this fails in the case where debug info is only enabled partially when -fprofile-sample-use is active. The reason llvm.dbg.cu is not present in these cases is to prevent codegen from emitting debug info (as it is only used for the sample profile pass). This changes the discriminators pass to also emit discriminators even when debug info is not being emitted. llvm-svn: 252763
* [SLP] Enable -slp-vectorize-hor by default.Charlie Turner2015-11-111-1/+1
| | | | | | | | | | | Measurements primarily on AArch64 have shown this feature does not significantly effect compile-time. The are no significant perf changes in LNT, but for AArch64 at least, there are wins in third party benchmarks. As discussed on llvm-dev, we're going to try turning this on by default and see how other targets react to the change. llvm-svn: 252733
* [ASan] Enable optional ASan recovery.Yury Gribov2015-11-111-18/+33
| | | | | | Differential Revision: http://reviews.llvm.org/D14242 llvm-svn: 252719
* Revert "Strip metadata when speculatively hoisting instructions"Renato Golin2015-11-102-9/+0
| | | | | | | This reverts commit r252604, as it broke all ARM and AArch64 buildbots, as well as some x86, et al. llvm-svn: 252623
* Strip metadata when speculatively hoisting instructionsIgor Laevsky2015-11-102-0/+9
| | | | | | | | | | | | | | | | This is fix for PR24059. When we are hoisting instruction above some condition it may turn out that metadata on this instruction was control dependant on the condition. This metadata becomes invalid and we need to drop it. This patch should cover most obvious places of speculative execution (which I have found by greping isSafeToSpeculativelyExecute). I think there are more cases but at least this change covers the severe ones. Differential Revision: http://reviews.llvm.org/D14398 llvm-svn: 252604
* [sanitizer] Use same shadow offset for ASAN on aarch64Adhemerval Zanella2015-11-091-15/+2
| | | | | | | | | This patch makes ASAN for aarch64 use the same shadow offset for all currently supported VMAs (39 and 42 bits). The shadow offset is the same for 39-bit (36). Similar to ppc64 port, aarch64 transformation also requires to use an add instead of 'or' for 42-bit VMA. llvm-svn: 252495
* Add discriminators for call instructions that are from the same line and ↵Dehao Chen2015-11-091-0/+32
| | | | | | | | | | | | | | same basic block. Summary: Call instructions that are from the same line and same basic block needs to have separate discriminators to distinguish between different callsites. Reviewers: davidxl, dnovillo, dblaikie Subscribers: dblaikie, probinson, llvm-commits Differential Revision: http://reviews.llvm.org/D14464 llvm-svn: 252492
* Simplify. NFC.Chad Rosier2015-11-091-4/+1
| | | | llvm-svn: 252491
* GlobalOpt should maintain externally_initialized when splitting aggregatesOliver Stannard2015-11-091-0/+2
| | | | | | | | | | | | | When GlobalOpt splits an internal, global variable with an aggregate type, it should propagate the externally_initialized flag to the newly created globals. This makes the pass safe for our downstream use of this flag, while still allowing some useful optimisations (such as removing dead parts of the split aggregate) to be performed. Differential Revision: http://reviews.llvm.org/D13382 llvm-svn: 252490
* [LoopVectorize] Address post-commit feedback on r250032James Molloy2015-11-091-3/+4
| | | | | | | | | | Implemented as many of Michael's suggestions as were possible: * clang-format the added code while it is still fresh. * tried to change Value* to Instruction* in many places in computeMinimumValueSizes - unfortunately there are several places where Constants need to be handled so this wasn't possible. * Reduce the pass list on loop-vectorization-factors.ll. * Fix a bug where we were querying MinBWs for I->getOperand(0) but using MinBWs[I]. llvm-svn: 252469
* Allow LLE/LD and the loop versioning infrastructure to use SCEV predicatesSilviu Baranga2015-11-093-28/+90
| | | | | | | | | | | | | | | | | | | Summary: LAA currently generates a set of SCEV predicates that must be checked by users. In the case of Loop Distribute/Loop Load Elimination, no such predicates could have been emitted, since we don't allow stride versioning. However, in the future there could be SCEV predicates that will need to be checked. This change adds support for SCEV predicate versioning in the Loop Distribute, Loop Load Eliminate and the loop versioning infrastructure. Reviewers: anemet Subscribers: mssimpso, sanjoy, llvm-commits Differential Revision: http://reviews.llvm.org/D14240 llvm-svn: 252467
* [LoopStrengthReduce] Don't bother fixing up PHIs from EH Pad predsDavid Majnemer2015-11-081-0/+3
| | | | | | | | We cannot really insert fixup code into a PHI's predecessor. This fixes PR25445. llvm-svn: 252416
* Unbreak the buildSanjoy Das2015-11-071-1/+1
| | | | | | | My code clashed with some ilist iterator changes upstream. Fix by adding an explicit "&*" coercion. llvm-svn: 252392
* [FunctionAttrs] Add comment and clarify assertion message; NFCSanjoy Das2015-11-071-1/+6
| | | | llvm-svn: 252389
* [FunctionAttrs] Add handling for operand bundlesSanjoy Das2015-11-071-4/+31
| | | | | | | | | | | | | | Summary: Teach the FunctionAttrs to do the right thing for IR with operand bundles. Reviewers: reames, chandlerc Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D14408 llvm-svn: 252387
* [FunctionAttrs] Fix an iterator wraparound bugSanjoy Das2015-11-071-18/+19
| | | | | | | | | | | | | | | | | | | Summary: This change fixes an iterator wraparound bug in `determinePointerReadAttrs`. Ideally, ++'ing off the `end()` of an iplist should result in a failed assert, but currently iplist seems to silently wrap to the head of the list on `end()++`. This is why the bad behavior is difficult to demonstrate. Reviewers: chandlerc, reames Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D14350 llvm-svn: 252386
* [InstCombine] Teach FoldPHIArgZextsIntoPHI about EHPadsDavid Majnemer2015-11-071-0/+6
| | | | | | | | FoldPHIArgZextsIntoPHI cannot insert an instruction after the PHI if there is an EHPad in the BB. Doing so would result in an instruction inserted after a terminator. llvm-svn: 252377
* ADT: Remove last implicit ilist iterator conversions, NFCDuncan P. N. Exon Smith2015-11-073-6/+7
| | | | | | | | | | Some implicit ilist iterator conversions have crept back into Analysis, Transforms, Hexagon, and llvm-stress. This removes them. I'll commit a patch immediately after this to disallow them (in a separate patch so that it's easy to revert if necessary). llvm-svn: 252371
* [InstCombine] Don't insert an instruction after a terminatorDavid Majnemer2015-11-061-0/+6
| | | | | | | | We tried to insert a cast of a phi in a block whose terminator is an EHPad. This is invalid. Do not attempt the transform in these circumstances. llvm-svn: 252370
* Add 'notail' marker for call instructions.Akira Hatanaka2015-11-061-2/+4
| | | | | | | | | | | | This marker prevents optimization passes from adding 'tail' or 'musttail' markers to a call. Is is used to prevent tail call optimization from being performed on the call. rdar://problem/22667622 Differential Revision: http://reviews.llvm.org/D12923 llvm-svn: 252368
* [InstCombine] Don't RAUW tokens with undefDavid Majnemer2015-11-061-2/+3
| | | | | | Let SimplifyCFG remove unreachable BBs which define token instructions. llvm-svn: 252343
* [SimplifyLibCalls] Don't hardcode the function name.Davide Italiano2015-11-061-1/+2
| | | | llvm-svn: 252342
* Fix SLPVectorizer commutativity reorderingMehdi Amini2015-11-061-76/+69
| | | | | | | | | | | | | | | | | | | The SLPVectorizer had a very crude way of trying to benefit from associativity: it tried to optimize for splat/broadcast or in order to have the same operator on the same side. This is benefitial to the cost model and allows more vectorization to occur. This patch improve the logic and make the detection optimal (locally, we don't look at the full tree but only at the immediate children). Should fix https://llvm.org/bugs/show_bug.cgi?id=25247 Reviewers: mzolotukhin Differential Revision: http://reviews.llvm.org/D13996 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 252337
* [ValueTracking] Add parameters to isImpliedCondition; NFCSanjoy Das2015-11-062-2/+4
| | | | | | | | | | | | | | | | Summary: This change makes the `isImpliedCondition` interface similar to the rest of the functions in ValueTracking (in that it takes a DataLayout, AssumptionCache etc.). This is an NFC, intended to make a later diff less noisy. Depends on D14369 Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D14391 llvm-svn: 252333
* [LIR] Simplify code by making DataLayout globally accessible. NFC.Chad Rosier2015-11-061-11/+10
| | | | llvm-svn: 252317
* DI: Reverse direction of subprogram -> function edge.Peter Collingbourne2015-11-056-52/+24
| | | | | | | | | | | | | | | | | | | | | | | Previously, subprograms contained a metadata reference to the function they described. Because most clients need to get or set a subprogram for a given function rather than the other way around, this created unneeded inefficiency. For example, many passes needed to call the function llvm::makeSubprogramMap() to build a mapping from functions to subprograms, and the IR linker needed to fix up function references in a way that caused quadratic complexity in the IR linking phase of LTO. This change reverses the direction of the edge by storing the subprogram as function-level metadata and removing DISubprogram's function field. Since this is an IR change, a bitcode upgrade has been provided. Fixes PR23367. An upgrade script for textual IR for out-of-tree clients is attached to the PR. Differential Revision: http://reviews.llvm.org/D14265 llvm-svn: 252219
OpenPOWER on IntegriCloud