summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
* [asan] experimental tracing for indirect calls, llvm part.Kostya Serebryany2014-10-271-4/+44
| | | | llvm-svn: 220699
* InstCombine: Fix a combine assuming that icmp operands were integersDavid Majnemer2014-10-271-0/+4
| | | | | | | | | An icmp may have pointer arguments, it isn't limited to integers or vectors of integers. This fixes PR21388. llvm-svn: 220664
* Add an option to the LTO code generator to disable vectorization during LTOArnold Schwaighofer2014-10-261-1/+1
| | | | | | | | | | | | | | | | | | | | We used to always vectorize (slp and loop vectorize) in the LTO pass pipeline. r220345 changed it so that we used the PassManager's fields 'LoopVectorize' and 'SLPVectorize' out of the desire to be able to disable vectorization using the cl::opt flags 'vectorize-loops'/'slp-vectorize' which the before mentioned fields default to. Unfortunately, this turns off vectorization because those fields default to false. This commit adds flags to the LTO library to disable lto vectorization which reconciles the desire to optionally disable vectorization during LTO and the desired behavior of defaulting to enabled vectorization. We really want tools to set PassManager flags directly to enable/disable vectorization and not go the route via cl::opt flags *in* PassManagerBuilder.cpp. llvm-svn: 220652
* LSR: Minor cleanup after Daniel's patch.Andrew Trick2014-10-251-4/+2
| | | | | | Combine the Inserted an Done sets into a Visited set. llvm-svn: 220623
* Fix LSR compile time.Andrew Trick2014-10-251-0/+5
| | | | | | | | | | | | This is a simple fix that brings the compilation time from 5min to 5s on a specific real-world example. It's a large chain of computation in a crypto routine (always a problem for SCEV). A unit test is not feasible and there would be no way to check it. The fix is just basic good practice for dealing with SCEVs, there's no risk of regression. Patch by Daniel Reynaud! llvm-svn: 220622
* [SeparateConstOffsetFromGEP] Fixed a bug related to unsigned moduloJingyue Wu2014-10-251-6/+6
| | | | | | | | | | | The dividend in "signed % unsigned" is treated as unsigned instead of signed, causing unexpected behavior such as -64 % (uint64_t)24 == 0. Added a regression test in split-gep.ll Patched by Hao Liu. llvm-svn: 220618
* Clean up assume intrinsic pattern matching, no need to check that the ↵Benjamin Kramer2014-10-251-1/+1
| | | | | | | | argument is a value. Also make it const safe and remove superfluous casting. NFC. llvm-svn: 220616
* [SeparateConstOffsetFromGEP] Fixed a bug in rebuilding OR expressionsJingyue Wu2014-10-251-2/+7
| | | | | | | | | | | | The two operands of the new OR expression should be NextInChain and TheOther instead of the two original operands. Added a regression test in split-gep.ll. Hao Liu reported this bug, and provded the test case and an initial patch. Thanks! llvm-svn: 220615
* InstCombine: Remove overzealous assertsDavid Majnemer2014-10-251-12/+19
| | | | | | | | | | | | | | These asserts can trigger if the worklist iteration order is sufficiently unlucky. Instead of adding special case logic to handle these edge conditions, just bail out on trying to transform them: InstSimplify will get them when it reaches them on the worklist. This fixes PR21378. N.B. No test case is included because any test would rely on the fragile worklist iteration order. llvm-svn: 220612
* [msan] Make -msan-check-constant-shadow a bit stronger.Evgeniy Stepanov2014-10-241-3/+10
| | | | | | Allow (under the experimental flag) non-Instructions to participate in MSan checks. llvm-svn: 220601
* If requested, apply function merging at -O0 too. It's useful there to reduce ↵Nick Lewycky2014-10-231-6/+10
| | | | | | the time to compile. llvm-svn: 220537
* Make getDISubprogram(const Function *F) available in LLVMTimur Iskhodzhanov2014-10-231-24/+1
| | | | | | Reviewed at http://reviews.llvm.org/D5950 llvm-svn: 220536
* Handle sqrt() shrinking in SimplifyLibCalls like any other callSanjay Patel2014-10-232-41/+14
| | | | | | | | | | | | | | | | | This patch removes a chunk of special case logic for folding (float)sqrt((double)x) -> sqrtf(x) in InstCombineCasts and handles it in the mainstream path of SimplifyLibCalls. No functional change intended, but I loosened the restriction on the existing sqrt testcases to allow for this optimization even without unsafe-fp-math because that's the existing behavior. I also added a missing test case for not shrinking the llvm.sqrt.f64 intrinsic in case the result is used as a double. Differential Revision: http://reviews.llvm.org/D5919 llvm-svn: 220514
* Assert that ValueHandleBase::ValueIsRAUWd doesn't change the tracked Value type.Frederic Riss2014-10-231-3/+9
| | | | | | | | | | | This invariant is enforced in Value::replaceAllUsesWith, thus it seems logical to apply it also to ValueHandles. This commit fixes InstCombine to not trigger the assertion during the removal of constant bitcasts in call instructions. Differential Revision: http://reviews.llvm.org/D5828 llvm-svn: 220468
* [msan] Emit checks for constant shadow values under an experimental flag.Evgeniy Stepanov2014-10-231-3/+12
| | | | | | Does not change the default behavior. llvm-svn: 220457
* LoopVectorize: Simplify code. No functionality change.Benjamin Kramer2014-10-221-19/+7
| | | | llvm-svn: 220405
* Shorten auto iterators for function basic blocks.Diego Novillo2014-10-221-37/+36
| | | | | | | | Use consistent naming for basic block instances. No functional changes. llvm-svn: 220404
* Use auto iteration in lib/Transforms/Scalar/SampleProfile.cpp. No functional ↵Diego Novillo2014-10-221-18/+15
| | | | | | changes. llvm-svn: 220394
* Preserving 'nonnull' metadata in SimplifyCFGPhilip Reames2014-10-221-1/+4
| | | | | | | | | | When we hoist two loads above an if, we can preserve the nonnull metadata. We could also do the same for sinking them, but we appear to not handle metadata at all in that case. Thanks to Hal for the review. Differential Revision: http://reviews.llvm.org/D5910 llvm-svn: 220392
* Shrinkify libcalls: use float versions of double libm functions with ↵Sanjay Patel2014-10-222-10/+24
| | | | | | | | | | | | | | | | | | | | | | | | fast-math (bug 17850) When a call to a double-precision libm function has fast-math semantics (via function attribute for now because there is no IR-level FMF on calls), we can avoid fpext/fptrunc operations and use the float version of the call if the input and output are both float. We already do this optimization using a command-line option; this patch just adds the ability for fast-math to use the existing functionality. I moved the cl::opt from InstructionCombining into SimplifyLibCalls because it's only ever used internally to that class. Modified the existing test cases to use the unsafe-fp-math attribute rather than repeating all tests. This patch should solve: http://llvm.org/bugs/show_bug.cgi?id=17850 Differential Revision: http://reviews.llvm.org/D5893 llvm-svn: 220390
* Change error to warning when a profile cannot be found.Diego Novillo2014-10-221-1/+3
| | | | | | | | When the profile for a function cannot be applied, we use to emit an error. This seems extreme. The compiler can continue, it's just that the optimization opportunities won't include profile information. llvm-svn: 220386
* Support using sample profiles with partial debug info.Diego Novillo2014-10-221-12/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: When using a profile, we used to require the use -gmlt so that we could get access to the line locations. This is used to match line numbers in the input profile to the line numbers in the function's IR. But this is actually not necessary. The driver can provide source location tracking without the emission of debug information. In these cases, the annotation 'llvm.dbg.cu' is missing from the IR, but the actual line location annotations are still present. This patch adds a new way of looking for the start of the current function. Instead of looking through the compile units in llvm.dbg.cu, we can walk up the scope for the first instruction in the function with a debug loc. If that describes the function, we use it. Otherwise, we keep looking until we find one. If no such instruction is found, we then give up and produce an error. Reviewers: echristo, dblaikie Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D5887 llvm-svn: 220382
* [msan] Handle param-tls overflow.Evgeniy Stepanov2014-10-221-14/+34
| | | | | | | | ParamTLS (shadow for function arguments) is of limited size. This change makes all arguments that do not fit unpoisoned, and avoids writing past the end of a TLS buffer. llvm-svn: 220351
* Revert "Teach the load analysis to allow finding available values which ↵Hans Wennborg2014-10-212-6/+5
| | | | | | | | require" (r220277) This seems to have caused PR21330. llvm-svn: 220349
* LTO: respect command-line options that disable vectorization.JF Bastien2014-10-211-2/+4
| | | | | | | | | | | | Summary: Patches 202051 and 208013 added calls to LTO's PassManager which unconditionally add LoopVectorizePass and SLPVectorizerPass instead of following the logic in PassManagerBuilder::populateModulePassManager and honoring the -vectorize-loops -run-slp-after-loop-vectorization flags. Reviewers: nadav, aschwaighofer, yijiang Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D5884 llvm-svn: 220345
* Add minnum / maxnum intrinsicsMatt Arsenault2014-10-212-0/+86
| | | | | | | | | | | | These are named following the IEEE-754 names for these functions, rather than the libm fmin / fmax to avoid possible ambiguities. Some languages may implement something resembling fmin / fmax which return NaN if either operand is to propagate errors. These implement the IEEE-754 semantics of returning the other operand if either is a NaN representing missing data. llvm-svn: 220341
* Teach combineMetadata how to merge 'nonnull' metadata.Philip Reames2014-10-211-0/+4
| | | | | | combineMetadata is used when merging two instructions into one. This change teaches it how to merge 'nonnull' - i.e. only preserve it on the new instruction if it's set on both sources. This isn't actually used yet since I haven't adjusted any of the call sites to pass in nonnull as a 'known metadata'. llvm-svn: 220325
* Preserve 'nonnull' when changing type of the load.Philip Reames2014-10-211-0/+1
| | | | | | | | When changing the type of a load in Chandler's recent InstCombine changes, we can preserve the new 'nonnull' metadata. I considered adding an assert since 'nonnull' is only valid on pointer types, but casting a pointer to a non-pointer would involve more than a bitcast anyways. If someone extends this transform to handle more than bitcasts, the verifier will report the malformed IR, so a separate assertion isn't needed. Also, the fpmath flags would have the same problem. llvm-svn: 220324
* InstCombine: Simplify FoldICmpCstShrCstDavid Majnemer2014-10-211-48/+16
| | | | | | | | | This function was complicated by the fact that it tried to perform canonicalizations that were already preformed by InstSimplify. Remove this extra code and move the tests over to InstSimplify. Add asserts to make sure our preconditions hold before we make any assumptions. llvm-svn: 220314
* Teach the load analysis to allow finding available values which requireChandler Carruth2014-10-212-5/+6
| | | | | | | | | | | | | | | | | | | | inttoptr or ptrtoint cast provided there is datalayout available. Eventually, the datalayout can just be required but in practice it will always be there today. To go with the ability to expose available values requiring a ptrtoint or inttoptr cast, helpers are added to perform one of these three casts. These smarts are necessary to finish canonicalizing loads and stores to the operational type requirements without regressing fundamental combines. I've added some test cases. These should actually improve as the load combining and store combining improves, but they may fundamentally be highlighting some missing combines for select in addition to exercising the specific added logic to load analysis. llvm-svn: 220277
* Do not attribute static allocas to the call site's DebugLoc.Paul Robinson2014-10-211-0/+6
| | | | | | | | | | | | | When functions are inlined, instructions without debug information are attributed to the call site's DebugLoc. After inlining, inlined static allocas are moved to the caller's entry block, adjacent to the caller's original static alloca instructions. By retaining the call site's DebugLoc, these instructions could cause instructions that were subsequently inserted at the entry block to pick up the same DebugLoc. Patch by Wolfgang Pieb! llvm-svn: 220255
* Introduce enum values for previously defined metadata types. (NFC)Philip Reames2014-10-212-7/+3
| | | | | | | | | | | Our metadata scheme lazily assigns IDs to string metadata, but we have a mechanism to preassign them as well. Using a preassigned ID is helpful since we get compile time type checking, and avoid some (minimal) string construction and comparison. This change adds enum value for three existing metadata types: + MD_nontemporal = 9, // "nontemporal" + MD_mem_parallel_loop_access = 10, // "llvm.mem.parallel_loop_access" + MD_nonnull = 11 // "nonnull" I went through an updated various uses as well. I made no attempt to get all uses; I focused on the ones which were easily grepable and easily to translate. For example, there were several items in LoopInfo.cpp I chose not to update. llvm-svn: 220248
* IR: Replace DataLayout::RoundUpAlignment with RoundUpToAlignmentDavid Majnemer2014-10-201-4/+4
| | | | | | No functional change intended, just cleaning up some code. llvm-svn: 220187
* Fix a somewhat subtle pair of issues with JumpThreading I introduced inChandler Carruth2014-10-201-3/+6
| | | | | | | | | | | | | | | r220178. First, the creation routine doesn't insert prior to the terminator of the basic block provided, but really at the end of the basic block. Instead, get the terminator and insert before that. The next issue was that we need to ensure multiple PHI node entries for a single predecessor re-use the same cast instruction rather than creating new ones. All of the logic here was without tests previously. I've reduced and added a test case from the test suite that crashed without both of these fixes. llvm-svn: 220186
* Teach the load analysis driving core instcombine logic and other bits ofChandler Carruth2014-10-202-2/+12
| | | | | | | | | | | | | | | | | | | logic to look through pointer casts, making them trivially stronger in the face of loads and stores with intervening pointer casts. I've included a few test cases that demonstrate the kind of folding instcombine can do without pointer casts and then variations which obfuscate the logic through bitcasts. Without this patch, the variations all fail to optimize fully. This is more important now than it has been in the past as I've started moving the load canonicialization to more closely follow the value type requirements rather than the pointer type requirements and thus this needs to be prepared for more pointer casts. When I made the same change to stores several test cases regressed without logic along these lines so I wanted to systematically improve matters first. llvm-svn: 220178
* Do a better and more complete job of preserving metadata when combiningChandler Carruth2014-10-191-8/+58
| | | | | | | | | | | | | | | | | | | | loads. This handles many more cases than just the AA metadata, some of them suggested by Hal in his review of the AA metadata handling patch. I've tried to test this behavior where tractable to do so. I'll point out that I have specifically *not* included a test for debuginfo because it was going to require 2 or 3 times as much work to craft some input which would survive the "helpful" stripping of debug info metadata that doesn't match the desired schema. This is another good example of why the current state of write-ability for our debug info metadata is unacceptable. I spent over 30 minutes trying to conjure some test case that would survive, even copying from other debug info tests, but it always failed to survive with no explanation of why or how I might fix it. =[ llvm-svn: 220165
* InstCombine: (sub (or A B) (xor A B)) --> (and A B)David Majnemer2014-10-191-0/+9
| | | | | | | | | | | The following implements the transformation: (sub (or A B) (xor A B)) --> (and A B). Patch by Ankur Garg! Differential Revision: http://reviews.llvm.org/D5719 llvm-svn: 220163
* InstCombine: Optimize icmp eq/ne (shl Const2, A), Const1David Majnemer2014-10-192-1/+50
| | | | | | | | | | | | | | | The following implements the optimization for sequences of the form: icmp eq/ne (shl Const2, A), Const1 Such sequences can be transformed to: icmp eq/ne A, (TrailingZeros(Const1) - TrailingZeros(Const2)) This handles only the equality operators for now. Other operators need to be handled. Patch by Ankur Garg! llvm-svn: 220162
* Fix a long-standing miscompile in the load analysis that was uncoveredChandler Carruth2014-10-191-1/+5
| | | | | | | | | | | | | | | | | | | by my refactoring of this code. The method isSafeToLoadUnconditionally assumes that the load will proceed with the preferred type alignment. Given that, it has to ensure that the alloca or global is at least that aligned. It has always done this historically when a datalayout is present, but has never checked it when the datalayout is absent. When I refactored the code in r220156, I exposed this path when datalayout was present and that turned the latent bug into a patent bug. This fixes the issue by just removing the special case which allows folding things without datalayout. This isn't worth the complexity of trying to tease apart when it is or isn't safe without actually knowing the preferred alignment. llvm-svn: 220161
* Preserve AA metadata when combining (cast (load (...))) -> (load (castChandler Carruth2014-10-181-0/+3
| | | | | | (...))). llvm-svn: 220141
* [InstCombine] Do an about-face on how LLVM canonicalizes (cast (loadChandler Carruth2014-10-181-72/+43
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ...)) and (load (cast ...)): canonicalize toward the former. Historically, we've tried to load using the type of the *pointer*, and tried to match that type as closely as possible removing as many pointer casts as we could and trading them for bitcasts of the loaded value. This is deeply and fundamentally wrong. Repeat after me: memory does not have a type! This was a hard lesson for me to learn working on SROA. There is only one thing that should actually drive the type used for a pointer, and that is the type which we need to use to load from that pointer. Matching up pointer types to the loaded value types is very useful because it minimizes the physical size of the IR required for no-op casts. Similarly, the only thing that should drive the type used for a loaded value is *how that value is used*! Again, this minimizes casts. And in fact, the *only* thing motivating types in any part of LLVM's IR are the types used by the operations in the IR. We should match them as closely as possible. I've ended up removing some tests here as they were testing bugs or behavior that is no longer present. Mostly though, this is just cleanup to let the tests continue to function as intended. The only fallout I've found so far from this change was SROA and I have fixed it to not be impeded by the different type of load. If you find more places where this change causes optimizations not to fire, those too are likely bugs where we are assuming that the type of pointers is "significant" for optimization purposes. llvm-svn: 220138
* [SROA] Change how SROA does vector-based promotion of allocas to handleChandler Carruth2014-10-181-44/+128
| | | | | | | | | | | | | | | | | | | | | | | | | | | | cases where the alloca type, the load types, and the store types used all disagree. Previously, the only way that vector-based promotion occured was if the alloca type was a vector type. This was one of the *very* few remaining uses of the alloca's type to guide SROA/mem2reg left in LLVM. It turns out it was a bad idea. The alloca type can change very easily based on the mixture of types loaded and stored to that alloca. We shouldn't be relying on it as a signal for very much. Instead, the source of truth should be loads and stores. We should canonicalize the loads and stores as much as possible and then rely on them exclusively in SROA. When looking and loads and stores, we may find many different candidate vector types. This change will let SROA try all of them to find a vector type which is a viable way to promote the entire alloca to a vector register. With this change, it becomes possible to do better canonicalization and optimization of loads and stores without breaking SROA in random ways, and that should allow fixing a core source of performance loss in hot numerical loops such as those in Eigen. llvm-svn: 220116
* [msan] Fix handling of byval arguments with large alignment.Evgeniy Stepanov2014-10-171-1/+2
| | | | | | | MSan param-tls slots are 8-byte aligned. This change clips alignment of memcpy into param-tls to 8. llvm-svn: 220101
* Revert "TRE: make TRE a bit more aggressive"Rafael Espindola2014-10-171-2/+8
| | | | | | | | | This reverts commit r219899. This also updates byval-tail-call.ll to make it clear what was breaking. Adding r219899 again will cause the load/store to disappear. llvm-svn: 220093
* [DSE] Remove no-data-layout-only type-based overlap checkingHal Finkel2014-10-171-8/+1
| | | | | | | | | | | | | | | | | DSE's overlap checking contained special logic, used only when no DataLayout was available, which inferred a complete overwrite when the pointee types were equal. This logic seems fine for regular loads/stores, but does not work for memcpy and friends. Instead of fixing this, I'm just removing it. Philosophically, transformations should not contain enhanced behavior used only when data layout is lacking (data layout should be strictly additive), and maintaining these rarely-tested code paths seems not worthwhile at this stage. Credit to Aliaksei Zasenka for the bug report and the diagnosis. The test case (slightly reduced from that provided by Aliaksei) replaces the original contents of test/Transforms/DeadStoreElimination/no-targetdata.ll -- a few other tests have been updated to have a data layout. llvm-svn: 220035
* [SROA] Switch the common variable name for the 'AllocaSlices' class toChandler Carruth2014-10-161-40/+42
| | | | | | | | | | | 'AS'. Using 'S' as this was a terrible idea. Arguably, 'AS' is not much better, but it at least follows the idea of using initialisms and removes active confusion about the AllocaSlices variable and a Slice variable. llvm-svn: 219963
* [SROA] More range-based cleanups to SROA, these brought to you byChandler Carruth2014-10-161-25/+12
| | | | | | | | | clang-modernize. I did have to clean up the variable types and whitespace a bit because the use of auto made the code much less readable here. llvm-svn: 219962
* [SROA] Switch a couple of overly complex iterator accessors to just beChandler Carruth2014-10-161-26/+10
| | | | | | | | | ArrayRef accessors. I think this even came up in review that this was over-engineered, and indeed it was. Time to un-build it. llvm-svn: 219958
* [SROA] Start more deeply moving SROA to use ranges rather than justChandler Carruth2014-10-161-45/+42
| | | | | | | | | | | | iterators. There are a ton of places where it essentially wants ranges rather than just iterators. This is just the first step that adds the core slice range typedefs and uses them in a couple of places. I still have to explicitly construct them because they've not been punched throughout the entire set of code. More range-based cleanups incoming. llvm-svn: 219955
* Allow call-slop optzn for destinations with a suitable dereferenceable attributeBjorn Steinbrink2014-10-161-14/+16
| | | | | | | | | | | | | | | Summary: Currently, call slot optimization requires that if the destination is an argument, the argument has the sret attribute. This is to ensure that the memory access won't trap. In addition to sret, we can also allow the optimization to happen for arguments that have the new dereferenceable attribute, which gives the same guarantee. Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D5832 llvm-svn: 219950
OpenPOWER on IntegriCloud