summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
* [sanitizer-coverage] make sure asan does not instrument coverage guards ↵Kostya Serebryany2016-11-151-1/+1
| | | | | | (reported in https://github.com/google/oss-fuzz/issues/84) llvm-svn: 287030
* Revert r286999 which caused buildbot test failures. Some testcases need to ↵Wei Mi2016-11-151-5/+6
| | | | | | be made target specific. llvm-svn: 287014
* [LSR] Allow formula containing Reg for SCEVAddRecExpr related with outerloop.Wei Mi2016-11-151-6/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In RateRegister of existing LSR, if a formula contains a Reg which is a SCEVAddRecExpr, and this SCEVAddRecExpr's loop is an outerloop, the formula will be marked as Loser and dropped. Suppose we have an IR that %for.body is outerloop and %for.body2 is innerloop. LSR only handle inner loop now so only %for.body2 will be handled. Using the logic above, formula like reg(%array) + reg({1,+, %size}<%for.body>) + 1*reg({0,+,1}<%for.body2>) will be dropped no matter what because reg({1,+, %size}<%for.body>) is a SCEVAddRecExpr type reg related with outerloop. Only formula like reg(%array) + 1*reg({{1,+, %size}<%for.body>,+,1}<nuw><nsw><%for.body2>) will be kept because the SCEVAddRecExpr related with outerloop is folded into the initial value of the SCEVAddRecExpr related with current loop. But in some cases, we do need to share the basic induction variable reg{0 ,+, 1}<%for.body2> among LSR Uses to reduce the final total number of induction variables used by LSR, so we don't want to drop the formula like reg(%array) + reg({1,+, %size}<%for.body>) + 1*reg({0,+,1}<%for.body2>) unconditionally. From the existing comment, it tries to avoid considering multiple level loops at the same time. However, existing LSR only handles innermost loop, so for any SCEVAddRecExpr with a loop other than current loop, it is an invariant and will be simple to handle, and the formula doesn't have to be dropped. Differential Revision: https://reviews.llvm.org/D26429 llvm-svn: 286999
* [IndVars] Change the order to compute WidenAddRec in widenIVUse.Wei Mi2016-11-151-2/+2
| | | | | | | | | | | | | | | | When both WidenIV::getWideRecurrence and WidenIV::getExtendedOperandRecurrence return non-null but different WideAddRec, if getWideRecurrence is called before getExtendedOperandRecurrence, we won't bother to call getExtendedOperandRecurrence again. But As we know it is possible that after SCEV folding, we cannot prove the legality using the SCEVAddRecExpr returned by getWideRecurrence. Meanwhile if getExtendedOperandRecurrence returns non-null WideAddRec, we know for sure that it is legal to do widening for current instruction. So it is better to put getExtendedOperandRecurrence before getWideRecurrence, which will increase the chance of successful widening. Differential Revision: https://reviews.llvm.org/D26059 llvm-svn: 286987
* [AVX-512] Add AVX-512 vector shift intrinsics to memory santitizer.Craig Topper2016-11-151-0/+31
| | | | | | Just needed to add the intrinsics to the exist switch. The code is generic enough to support the wider vectors with no changes. llvm-svn: 286980
* Revert "[JumpThreading] Unfold selects that depend on the same condition"Pablo Barrio2016-11-151-77/+38
| | | | | | This reverts commit ac54d0066c478a09c7cd28d15d0f9ff8af984afc. llvm-svn: 286976
* Revert "[JumpThreading] Prevent non-deterministic use lists"Pablo Barrio2016-11-151-7/+8
| | | | | | This reverts commit f2c2f5354070469dac253373c66527ca971ddc66. llvm-svn: 286975
* [LoopVectorizer] When estimating reg usage, unused insts may "end" another useRobert Lougher2016-11-151-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | The register usage algorithm incorrectly treats instructions whose value is not used within the loop (e.g. those that do not produce a value). The algorithm first calculates the usages within the loop. It iterates over the instructions in order, and records at which instruction index each use ends (in fact, they're actually recorded against the next index, as this is when we want to delete them from the open intervals). The algorithm then iterates over the instructions again, adding each instruction in turn to a list of open intervals. Instructions are then removed from the list of open intervals when they occur in the list of uses ended at the current index. The problem is, instructions which are not used in the loop are skipped. However, although they aren't used, the last use of a value may have been recorded against that instruction index. In this case, the use is not deleted from the open intervals, which may then bump up the estimated register usage. This patch fixes the issue by simply moving the "is used" check after the loop which erases the uses at the current index. Differential Revision: https://reviews.llvm.org/D26554 llvm-svn: 286969
* Test commit, remove trailing space.Florian Hahn2016-11-151-1/+1
| | | | | | This commit is used to test commit access. llvm-svn: 286957
* [tsan] Add support for C++ exceptions into TSan (call __tsan_func_exit ↵Kuba Brecka2016-11-145-63/+171
| | | | | | | | | | during unwinding), LLVM part This adds support for TSan C++ exception handling, where we need to add extra calls to __tsan_func_exit when a function is exitted via exception mechanisms. Otherwise the shadow stack gets corrupted (leaked). This patch moves and enhances the existing implementation of EscapeEnumerator that finds all possible function exit points, and adds extra EH cleanup blocks where needed. Differential Revision: https://reviews.llvm.org/D26177 llvm-svn: 286893
* [ThinLTO] Only promote exported locals as marked in indexTeresa Johnson2016-11-142-16/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: We have always speculatively promoted all renamable local values (except const non-address taken variables) for both the exporting and importing module. We would then internalize them back based on the ThinLink results if they weren't actually exported. This is inefficient, and results in unnecessary renames. It also meant we had to check the non-renamability of a value in the summary, which was already checked during function importing analysis in the ThinLink. Made renameModuleForThinLTO (which does the promotion/renaming) instead use the index when exporting, to avoid unnecessary renames/promotions. For importing modules, we can simply promoted all values as any local we import by definition is exported and needs promotion. This required changes to the method used by the FunctionImport pass (only invoked from 'opt' for testing) and when invoked from llvm-link, since neither does a ThinLink. We simply conservatively mark all locals in the index as promoted, which preserves the current aggressive promotion behavior. I also needed to change an llvm-lto based test where we had previously been aggressively promoting values that weren't importable (aliasees), but now will not promote. Reviewers: mehdi_amini Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D26467 llvm-svn: 286871
* [ThinLTO] Make inline assembly handling more efficient in summaryTeresa Johnson2016-11-141-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The change in r285513 to prevent exporting of locals used in inline asm added all locals in the llvm.used set to the reference set of functions containing inline asm. Since these locals were marked NoRename, this automatically prevented importing of the function. Unfortunately, this caused an explosion in the summary reference lists in some cases. In my particular example, it happened for a large protocol buffer generated C++ file, where many of the generated functions contained an inline asm call. It was exacerbated when doing a ThinLTO PGO instrumentation build, where the PGO instrumentation included thousands of private __profd_* values that were added to llvm.used. We really only need to include a single llvm.used local (NoRename) value in the reference list of a function containing inline asm to block it being imported. However, it seems cleaner to add a flag to the summary that explicitly describes this situation, which is what this patch does. Reviewers: mehdi_amini Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D26402 llvm-svn: 286840
* Remove redundant condition (PR28352) NFCI.Simon Pilgrim2016-11-141-2/+3
| | | | | | We were already testing is the op was not a leaf, so need to then test if it was a leaf (added it to the assert instead). llvm-svn: 286817
* [JumpThreading] Prevent non-deterministic use listsPablo Barrio2016-11-141-8/+7
| | | | | | | | | | | | | | | | | Summary: Unfolding selects was previously done with the help of a vector of pointers that was then sorted to be able to remove duplicates. As this sorting depends on the memory addresses, it was non-deterministic. A SetVector is used now so that duplicates are removed without the need of sorting first. Reviewers: mgrang, efriedma Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D26450 llvm-svn: 286807
* [InstCombine][AVX-512] Teach InstCombineCalls to handle the new unmasked ↵Craig Topper2016-11-131-4/+18
| | | | | | AVX-512 variable shift intrinsics. llvm-svn: 286755
* Bitcode: Change module reader functions to return an llvm::Expected.Peter Collingbourne2016-11-131-1/+4
| | | | | | Differential Revision: https://reviews.llvm.org/D26562 llvm-svn: 286752
* Analysis: Simplify the ScalarEvolution::getGEPExpr() interface. NFCI.Peter Collingbourne2016-11-132-6/+3
| | | | | | | | All existing callers were manually extracting information out of an existing GEP instruction and passing it to getGEPExpr(). Simplify the interface by changing it to take a GEPOperator instead. llvm-svn: 286751
* [InstCombine][AVX-512] Expand vector shift handling to work on the AVX-512 ↵Craig Topper2016-11-131-1/+45
| | | | | | | | shift by immediate and shift by single value. This does not include support for the AVX-512 variable shifts. That will be coming in a future patch. llvm-svn: 286739
* [InstCombine] use dyn_cast rather isa+cast; NFCSanjay Patel2016-11-111-2/+2
| | | | | | Follow-up to r286664 cleanup as suggested by Eli. Thanks! llvm-svn: 286671
* [InstCombine] clean up foldSelectOpOp(); NFCSanjay Patel2016-11-111-10/+4
| | | | llvm-svn: 286664
* [tsan][llvm] Implement the function attribute to disable TSan checking at ↵Anna Zaks2016-11-111-0/+23
| | | | | | | | | | | | | | | | | | | | | | | run time This implements a function annotation that disables TSan checking for the function at run time. The benefit over attribute((no_sanitize("thread"))) is that the accesses within the callees will also be suppressed. The motivation for this attribute is a guarantee given by the objective C language that the calls to the reference count decrement and object deallocation will be synchronized. To model this properly, we would need to intercept all ref count decrement calls (which are very common in ObjC due to use of ARC) and also every single message send. Instead, we propose to just ignore all accesses made from within dealloc at run time. The main downside is that this still does not introduce any synchronization, which means we might still report false positives if the code that relies on this synchronization is not executed from within dealloc. However, we have not seen this in practice so far and think these cases will be very rare. Differential Revision: https://reviews.llvm.org/D25858 llvm-svn: 286663
* [LV] Stop saying "use -Rpass-analysis=loop-vectorize"Adam Nemet2016-11-111-2/+1
| | | | | | | | | | | | | | | | | | This is PR28376. Unfortunately given the current structure of optimization diagnostics we lack the capability to tell whether the user has passed -Rpass-analysis=loop-vectorize since this is local to the front-end (BackendConsumer::OptimizationRemarkHandler). So rather than printing this even if the user has already passed -Rpass-analysis, this patch just punts and stops recommending this option. I don't think that getting this right is worth the complexity. Differential Revision: https://reviews.llvm.org/D26563 llvm-svn: 286662
* FunctionComparator: don't rely on argument evaluation order.Erik Eckstein2016-11-111-1/+3
| | | | | | | This is a follow-up on the recent refactoring of the FunctionMerge pass. It should fix a fail of the new FunctionComparator unittest whe compiling with MSVC. llvm-svn: 286648
* [cfi] Fix weak functions handling.Evgeniy Stepanov2016-11-111-2/+74
| | | | | | | | | | | | | | | When a function pointer is replaced with a jumptable pointer, special case is needed to preserve the semantics of extern_weak functions. Since a jumptable entry can not be extern_weak, we emulate that behaviour by replacing all references to F (the extern_weak function) with the following expression: F != nullptr ? JumpTablePtr : nullptr. Extra special care is needed for global initializers, since most (or probably all) backends can not lower an initializer that includes this kind of constant expression. Initializers like that are replaced with a global constructor (i.e. a runtime initializer). llvm-svn: 286636
* Make the FunctionComparator of the MergeFunctions pass a stand-alone utility.Erik Eckstein2016-11-113-1217/+924
| | | | | | | | | | | | | | | | | | | | | | | | | | | This is pure refactoring. NFC. This change moves the FunctionComparator (together with the GlobalNumberState utility) in to a separate file so that it can be used by other passes. For example, the SwiftMergeFunctions pass in the Swift compiler: https://github.com/apple/swift/blob/master/lib/LLVMPasses/LLVMMergeFunctions.cpp Details of the change: *) The big part is just moving code out of MergeFunctions.cpp into FunctionComparator.h/cpp *) Make FunctionComparator member functions protected (instead of private) so that a derived comparator class can use them. Following refactoring helps to share code between the base FunctionComparator class and a derived class: *) Add a beginCompare() function *) Move some basic function property comparisons into a separate function compareSignature() *) Do the GEP comparison inside cmpOperations() which now has a new needToCmpOperands reference parameter https://reviews.llvm.org/D25385 llvm-svn: 286632
* Fixed the lost FastMathFlags for FCmp operations in SLPVectorizer.Vyacheslav Klochkov2016-11-111-3/+4
| | | | | | | Reviewer: Michael Zolotukhin. Differential Revision: https://reviews.llvm.org/D26543 llvm-svn: 286626
* Bitcode: Change getModuleSummaryIndex() to return an llvm::Expected.Peter Collingbourne2016-11-111-35/+6
| | | | | | Differential Revision: https://reviews.llvm.org/D26539 llvm-svn: 286624
* [sancov] Don't instrument MSVC CRT stdio config helpersReid Kleckner2016-11-111-0/+5
| | | | | | | | They get called before initialization, which is a problem for winasan. Test coming in compiler-rt. llvm-svn: 286615
* [cfi] Implement cfi-icall using inline assembly.Evgeniy Stepanov2016-11-111-75/+159
| | | | | | | | | | | | | | | | | | | | | | | | | The current implementation is emitting a global constant that happens to evaluate to the same bytes + relocation as a jump instruction on X86. This does not work for PIE executables and shared libraries though, because we end up with a wrong relocation type. And it has no chance of working on ARM/AArch64 which use different relocation types for jump instructions (R_ARM_JUMP24) that is never generated for data. This change replaces the constant with module-level inline assembly followed by a hidden declaration of the jump table. Works fine for ARM/AArch64, but has some drawbacks. * Extra symbols are added to the static symbol table, which inflate the size of the unstripped binary a little. Stripped binaries are not affected. This happens because jump table declarations must be external (because their body is in the inline asm). * Original functions that were anonymous are now named <original name>.cfi, and it affects symbolization sometimes. This is necessary because the only user of these functions is the (inline asm) jump table, so they had to be added to @llvm.used, which does not allow unnamed functions. llvm-svn: 286611
* [InstCombine] fix formatting of FoldOpIntoSelect(); NFCISanjay Patel2016-11-111-41/+43
| | | | llvm-svn: 286604
* Add comments about why we put LoopSink pass at the very late stage.Dehao Chen2016-11-101-0/+4
| | | | llvm-svn: 286480
* [InstCombine] avoid infinite loop from shuffle-extract-insert sequence (PR30923)Sanjay Patel2016-11-101-0/+8
| | | | | | | | | | | | Removing the limitation in visitInsertElementInst() causes several regressions because we're not prepared to fold sequences of shuffles or inserts and extracts separated by shuffles. Fixing that appears to be a difficult mission because we are purposely trying to avoid creating shuffles with arbitrary shuffle masks because some targets may choke on those. https://llvm.org/bugs/show_bug.cgi?id=30923 llvm-svn: 286423
* Preserve assumption cache in loop-rotate.Eli Friedman2016-11-091-0/+4
| | | | | | | | | | | No testcase included because I can't figure out how to reduce it. (It's easy to write a testcase where rotation clones an assume, but that doesn't actually seem to trigger the crash in opt on its own; maybe an issue with the laziness?) Differential Revision: https://reviews.llvm.org/D26434 llvm-svn: 286410
* Minor unroll pass refacoring.Evgeny Stupachenko2016-11-091-35/+29
| | | | | | | | | | | | | | | | Summary: Unrolled Loop Size calculations moved to a function. Constant representing number of optimized instructions when "back edge" becomes "fall through" replaced with variable. Some comments added. Reviewers: mzolotukhin Differential Revision: http://reviews.llvm.org/D21719 From: Evgeny Stupachenko <evstupac@gmail.com> llvm-svn: 286389
* Bitcode: Change the materializer interface to return llvm::Error.Peter Collingbourne2016-11-091-8/+22
| | | | | | Differential Revision: https://reviews.llvm.org/D26439 llvm-svn: 286382
* Remove TimeValue usage from Scalar/SROA.cpp. NFC.Pavel Labath2016-11-091-2/+3
| | | | llvm-svn: 286361
* [ARM] Loop Strength Reduction crashes when targeting ARM or Thumb.Alexandros Lamprineas2016-11-091-3/+3
| | | | | | | | | | | Scalar Evolution asserts when not all the operands of an Add Recurrence Expression are loop invariants. Loop Strength Reduction should only create affine Add Recurrences, so that both the start and the step of the expression are loop invariants. Differential Revision: https://reviews.llvm.org/D26185 llvm-svn: 286347
* Enable Loop Sink pass for functions that has profile.Dehao Chen2016-11-092-4/+9
| | | | | | | | | | | | Summary: For functions with profile data, we are confident that loop sink will be optimal in sinking code. Reviewers: davidxl, hfinkel Subscribers: mehdi_amini, mzolotukhin, llvm-commits Differential Revision: https://reviews.llvm.org/D26155 llvm-svn: 286325
* [InstCombine] fix profitability equation for max-of-nots transformSanjay Patel2016-11-091-7/+6
| | | | | | | | | | As the test change shows, we can increase the critical path by adding a 'not' instruction, so make sure that we're actually removing an instruction if we do this transform. This transform could also cause us to miss folds of min/max pairs. llvm-svn: 286315
* [InstCombine] reduce indentation; NFCSanjay Patel2016-11-081-23/+20
| | | | llvm-svn: 286314
* [asan] Speed up compilation of large C++ stringmaps (tons of allocas) with ASanKuba Brecka2016-11-081-6/+12
| | | | | | | | This addresses PR30746, <https://llvm.org/bugs/show_bug.cgi?id=30746>. The ASan pass iterates over entry-block instructions and checks each alloca whether it's in NonInstrumentedStaticAllocaVec, which is apparently slow. This patch gathers the instructions to move during visitAllocaInst. Differential Revision: https://reviews.llvm.org/D26380 llvm-svn: 286296
* [LoopDistribute] Preserve GlobalsAA also in the new Pass Manager.Davide Italiano2016-11-081-0/+1
| | | | | | Differential Revision: https://reviews.llvm.org/D26408 llvm-svn: 286280
* [LibcallsShrinkWrap] This pass doesn't preserve the CFG.Davide Italiano2016-11-081-2/+5
| | | | | | | | | | For example, it invalidates the domtree, causing assertions in later passes which need dominator infos. Make it preserve GlobalsAA, as suggested by Eli. Differential Revision: https://reviews.llvm.org/D26381 llvm-svn: 286271
* Fix typo in comment. NFC.Chad Rosier2016-11-081-2/+2
| | | | llvm-svn: 286270
* Remove unused include. NFC.Chad Rosier2016-11-081-1/+0
| | | | llvm-svn: 286250
* Use the last 7 bits to represent the discriminator to fit it in 1 byte ↵Dehao Chen2016-11-081-5/+7
| | | | | | | | ULEB128 (NFC). From experiments, discriminator is rarely greater than 127. Here we enforce it to be no greater than 127 so that it will always fit in 1 byte. llvm-svn: 286245
* [JumpThreading] Unfold selects that depend on the same conditionPablo Barrio2016-11-081-38/+77
| | | | | | | | | | | | | | | | | | Summary: These are good candidates for jump threading. This enables later opts (such as InstCombine) to combine instructions from the selects with instructions out of the selects. SimplifyCFG will fold the select again if unfolding wasn't worth it. Patch by James Molloy and Pablo Barrio. Reviewers: rengolin, haicheng, sebpop Subscribers: jojo, jmolloy, llvm-commits Differential Revision: https://reviews.llvm.org/D26391 llvm-svn: 286236
* [TRE] Remove dead codeSanjoy Das2016-11-071-1/+0
| | | | | | Address review by Eli Friedman on rL286147. llvm-svn: 286165
* Reset debug loc to OldInduction in ↵Dehao Chen2016-11-071-1/+3
| | | | | | | | InnerLoopVectorizer::createInductionVariable. (NFC) This is to prevent SetInsertionPoint from setting debug loc to Latch->getTerminator(). llvm-svn: 286159
* Avoid tail recursion elimination across calls with operand bundlesSanjoy Das2016-11-071-1/+2
| | | | | | | | | | | | | | | | | | | | Summary: In some specific scenarios with well understood operand bundle types (like `"deopt"`) it may be possible to go ahead and convert recursion to iteration, but TailRecursionElimination does not have that logic today so avoid doing the right thing for now. I need some input on whether `"funclet"` operand bundles should also block tail recursion elimination. If not, I'll allow TRE across calls with `"funclet"` operand bundles and add a test case. Reviewers: rnk, majnemer, nlewycky, ahatanak Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D26270 llvm-svn: 286147
OpenPOWER on IntegriCloud