summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
* [InstCombine] Remove canonicalization for (X & C1) | C2 --> (X | C2) & ↵Craig Topper2017-04-031-11/+0
| | | | | | | | | | (C1|C2) when C1 & C2 have common bits. It turns out that SimplifyDemandedInstructionBits will get called earlier and remove bits from C1 first. Effectively doing (X & (C1&C2)) | C2. So by the time it got to this check there could be no common bits. I think the DAGCombiner has the same check but its check can be executed because it handles demanded bits later. I'll look at it next. llvm-svn: 299384
* [DAGCombine][InstCombine] Fix inverted if condition in equivalent comments ↵Craig Topper2017-04-031-1/+1
| | | | | | in DAGCombine and InstCombine. NFC llvm-svn: 299378
* Revert r299337 "[InstCombine] Remove redundant combine from visitAnd"Craig Topper2017-04-032-0/+89
| | | | | | | | One of the tsan bots started failing at this commit. I don't see anything obviously wrong with the commit so trying this to see if it recovers. Failing log: http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-autoconf/builds/6792 llvm-svn: 299366
* [InstCombine] fix formatting for foldLogOpOfMaskedICmps and related bits; NFCISanjay Patel2017-04-031-145/+158
| | | | | | | | | | | | 1. Improve enum, function, and variable names. 2. Improve comments. 3. Fix variable capitalization. 4. Run clang-format. As an existing code comment suggests, this should work with vector types / splat constants too, so making this look right first will reduce the diffs needed for that change. llvm-svn: 299365
* [APInt] Move isMask and isShiftedMask out of APIntOps and into the APInt ↵Craig Topper2017-04-031-1/+1
| | | | | | | | | | class. Implement them without memory allocation for multiword This moves the isMask and isShiftedMask functions to be class methods. They now use the MathExtras.h function for single word size and leading/trailing zeros/ones or countPopulation for the multiword size. The previous implementation made multiple temorary memory allocations to do the bitwise arithmetic operations to match the MathExtras.h implementation. Differential Revision: https://reviews.llvm.org/D31565 llvm-svn: 299362
* [InstCombine] Make foldOpWithConstantIntoOperand take a BinaryOperator ↵Craig Topper2017-04-032-2/+2
| | | | | | | | instead of a generic Instruction. It blindly assumes there are two operands so make it explicit. llvm-svn: 299351
* [InstCombine] Remove a And transform that should be handled by ↵Craig Topper2017-04-031-7/+0
| | | | | | SimplifyDemandedInstructionBits. NFCI llvm-svn: 299349
* [InstCombine] Make InstCombiner::OptAndOp take a BinaryOperator instead of ↵Craig Topper2017-04-022-4/+4
| | | | | | | | | | an Instruction. The callers have already performed the necessary cast before calling. This allows us to remove a comment that says the instruction must be a BinaryOperator and make it explicit in the argument type. Had to add a default case to the switch because BinaryOperator::getOpcode() returns a BinaryOps enum. llvm-svn: 299339
* [InstCombine] Remove redundant combine from visitAndCraig Topper2017-04-022-89/+0
| | | | | | | | | | As far as I can tell this combine is fully handled by SimplifyDemandedInstructionBits. I was only looking at this because it is the only user of APIntOps::isShiftedMask which is itself broken. As demonstrated by r299187. I was going to fix isShiftedMask and needed to make sure we had coverage for the new cases it would expose to this combine. But looks like we can nuke it instead. Differential Revision: https://reviews.llvm.org/D31543 llvm-svn: 299337
* NewGVN: Handle coercion of constant stores, loads, memory insts.Daniel Berlin2017-04-021-1/+93
| | | | | | | | | | | | | | | | | Summary: Depends on D30928. This adds support for coercion of stores and memory instructions that do not require insertion to process. Another few tests down. I added the relevant tests from rle.ll Reviewers: davide Subscribers: llvm-commits, Prazek Differential Revision: https://reviews.llvm.org/D30929 llvm-svn: 299330
* [BypassSlowDivision] Do not bypass division of hash-like valuesNikolai Bozhenov2017-04-021-12/+81
| | | | | | | | | | | | | | | | | Disable bypassing if one of the operands looks like a hash value. Slow division often occurs in hashtable implementations and fast division is never taken there because a hash value is extremely unlikely to have enough upper bits set to zero. A value is considered to be hash-like if it is produced by 1) XOR operation 2) Multiplication by a constant wider than the shorter type 3) PHI node with all incoming values being hash-like Differential Revision: https://reviews.llvm.org/D28200 llvm-svn: 299329
* MemorySSA: Add support for caching clobbering access in storesDaniel Berlin2017-04-022-9/+9
| | | | | | | | | | | | | | | | Summary: This enables us to cache the clobbering access for stores, despite the fact that we can't rewrite the use-def chains themselves. Early testing shows that, after this change, for larger testcases, it will be a significant net positive (memory and time) to remove the walker caching. Reviewers: george.burgess.iv, davide Subscribers: Prazek, llvm-commits Differential Revision: https://reviews.llvm.org/D31567 llvm-svn: 299322
* NewGVN: Don't try to kill off the stored value of stores whenDaniel Berlin2017-04-011-9/+22
| | | | | | | | | | | | | processing the congruence class of the store. Because we use the stored value of a store as the def, it isn't dead just because it appears as a def when it comes from a store. Note: I have not hit any cases with the memory code as it is where this breaks anything, just because of what memory congruences we actually allow. In a followup that improves memory congruence, this bug actually breaks real stuff (but the verifier catches it). llvm-svn: 299300
* NewGVN: Clean up GVNExpression memory hierarchy, restructure hash ↵Daniel Berlin2017-04-011-16/+8
| | | | | | computation a bit so we don't have to redefine it for loads, stores, and calls llvm-svn: 299299
* NewGVN: Use def_chain iterator in singleReachablePhiPath instead of recursionDaniel Berlin2017-04-011-20/+24
| | | | llvm-svn: 299298
* Move def_chain iterator to MemorySSA.h so it can be reusedDaniel Berlin2017-04-011-36/+0
| | | | llvm-svn: 299297
* MemorySSA: Push const correctness further.Daniel Berlin2017-04-011-8/+10
| | | | llvm-svn: 299295
* MemorySSA: Kill the WalkTargetCache now that we have getBlockDefs.Daniel Berlin2017-04-011-39/+6
| | | | llvm-svn: 299294
* [APInt] Fix bugs in isShiftedMask to match behavior of the similar function ↵Craig Topper2017-03-311-1/+1
| | | | | | | | | | in MathExtras.h This removes a parameter from the routine that was responsible for a lot of the issue. It was a bit count that had to be set to the BitWidth of the APInt and would get passed to getLowBitsSet. This guaranteed the call to getLowBitsSet would create an all ones value. This was then compared to (V | (V-1)). So the only shifted masks we detected had to have the MSB set. The one in tree user is a transform in InstCombine that never fires due to earlier transforms covering the case better. I've submitted a patch to remove it completely, but for now I've just adapted it to the new interface for isShiftedMask. llvm-svn: 299273
* [InstCombine] When adding an Instruction and its Users to the worklist at ↵Craig Topper2017-03-311-2/+2
| | | | | | | | the same time, make sure we put the Users in first. Then put in the instruction. This way we ensure we immediately revisit the instruction and do any additional optimizations before visiting the users. Otherwise we might visit the users, then the instruction, then users again, then instruction again. llvm-svn: 299267
* [APInt] Remove shift functions from APIntOps namespace. Replace the few ↵Craig Topper2017-03-311-5/+5
| | | | | | users with the APInt class methods. NFCI llvm-svn: 299248
* Do not translate rint into nearbyint, but truncate it like nearbyint.Joerg Sonnenberger2017-03-312-1/+3
| | | | | | | | | | | | | | A common way to implement nearbyint is by fiddling with the floating point environment and calling rint. This is used at least by the BSD libm and musl. As such, canonicalizing the latter to the former will create infinite loops for libm and generally pessimize performance, at least when the generic C versions are used. This change preserves the rint in the libcall translation and also handles the domain truncation logic, so that rint with float argument will be reduced to rintf etc. llvm-svn: 299247
* Fix the InstCombine to reserve the VP metadata and sets correct call count.Dehao Chen2017-03-311-0/+6
| | | | | | | | | | | | | | Summary: Currently the VP metadata was dropped when InstCombine converts a call to direct call. This patch converts the VP metadata to branch_weights so that its hotness is recorded. Reviewers: eraman, davidxl Reviewed By: davidxl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D31344 llvm-svn: 299228
* [Scalarizer] Handle scalar arguments in vector GEPMikael Holmen2017-03-311-3/+16
| | | | | | | | | | | | | | | | | | Summary: Triggered by commit r298620: "[LV] Vectorize GEPs". If we encounter a vector GEP with scalar arguments, we splat the scalar into a vector of appropriate size before we scatter the argument. Reviewers: arsenm, mehdi_amini, bkramer Reviewed By: arsenm Subscribers: bjope, mssimpso, wdng, llvm-commits Differential Revision: https://reviews.llvm.org/D31416 llvm-svn: 299186
* ThinLTOBitcodeWriter: Use Module::global_values(). NFCI.Peter Collingbourne2017-03-301-25/+7
| | | | llvm-svn: 299132
* [InstCombine] Fix typo last->least. NFCCraig Topper2017-03-301-3/+3
| | | | llvm-svn: 299123
* AMDGPU: Add all atomicrmw fields to atomic.inc/decMatt Arsenault2017-03-301-2/+8
| | | | | | Add scope, order, isVolatile llvm-svn: 299122
* [SimplifyIndvar] Replace the sdiv used by IV if we can prove both of its ↵Hongbin Zheng2017-03-301-4/+38
| | | | | | | | | | operands are non-negative Since there is no sdiv in SCEV, an 'udiv' is a better canonical form than an 'sdiv' as the user of induction variable Differential Revision: https://reviews.llvm.org/D31488 llvm-svn: 299118
* Spelling mistakes in comments. NFCI.Simon Pilgrim2017-03-301-1/+1
| | | | | | Based on corrections mentioned in patch for clang for PR27635 llvm-svn: 299072
* [InstCombine] Correct the check for vector GEPsMatthew Simpson2017-03-291-5/+5
| | | | | | | | | | | | | | Some of the GEP combines (e.g., descaling) can't handle vector GEPs. We have an existing check that attempts to bail out if given a vector GEP. However, the check only tests the GEP's pointer operand. A GEP results in a vector of pointers if at least one of its operands is vector-typed (e.g., its pointer operand could be a scalar, but its index could be a vector). We should just check the type of the GEP itself. This should fix PR32414. Reference: https://bugs.llvm.org/show_bug.cgi?id=32414 Differential Revision: https://reviews.llvm.org/D31470 llvm-svn: 299017
* Cleanup in preparation for D30703. NFCIFilipe Cabecinhas2017-03-291-27/+22
| | | | | | Make the enumerators follow the coding convention and start with OW_... llvm-svn: 298996
* [InstCombine] For select rule, use positive check of constant int for select ↵Anna Thomas2017-03-281-1/+1
| | | | | | operand. NFCI llvm-svn: 298906
* Revert "[asan] Delay creation of asan ctor."Alex Shlyapnikov2017-03-272-26/+21
| | | | | | | | Speculative revert. Some libfuzzer tests are affected. This reverts commit r298731. llvm-svn: 298890
* Revert "[asan] Put ctor/dtor in comdat."Alex Shlyapnikov2017-03-271-41/+9
| | | | | | | | Speculative revert, some libfuzzer tests are affected. This reverts commit r298756. llvm-svn: 298889
* [LV] Transform truncations of non-primary induction variablesMatthew Simpson2017-03-271-11/+10
| | | | | | | | | | | | The vectorizer tries to replace truncations of induction variables with new induction variables having the smaller type. After r295063, this optimization was applied to all integer induction variables, including non-primary ones. When optimizing the truncation of a non-primary induction variable, we still need to transform the new induction so that it has the correct start value. This should fix PR32419. Reference: https://bugs.llvm.org/show_bug.cgi?id=32419 llvm-svn: 298882
* [InstCombine] Avoid incorrect folding of select into phi nodes when incoming ↵Anna Thomas2017-03-271-1/+5
| | | | | | | | | | | | | | | | | | | | | | element is a vector type Summary: We are incorrectly folding selects into phi nodes when the incoming value of a phi node is a constant vector. This optimization is done in `FoldOpIntoPhi` when the select condition is a phi node with constant incoming values. Without the fix, we are miscompiling (i.e. incorrectly folding the select into the phi node) when the vector contains non-zero elements. This patch fixes the miscompile and we will correctly fold based on the select vector operand (see added test cases). Reviewers: majnemer, sanjoy, spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D31189 llvm-svn: 298845
* [LoopUnroll] Remap references in peeled iterationSerge Pavlov2017-03-261-4/+5
| | | | | | | | | References in cloned blocks must be remapped prior to dominator calculation. Differential Revision: https://reviews.llvm.org/D31281 llvm-svn: 298811
* Split the SimplifyCFG pass into two variants.Joerg Sonnenberger2017-03-264-23/+79
| | | | | | | | | | | | | | | | | | | | | | | The first variant contains all current transformations except transforming switches into lookup tables. The second variant contains all current transformations. The switch-to-lookup-table conversion results in code that is more difficult to analyze and optimize by other passes. Most importantly, it can inhibit Dead Code Elimination. As such it is often beneficial to only apply this transformation very late. A common example is inlining, which can often result in range restrictions for the switch expression. Changes in execution time according to LNT: SingleSource/Benchmarks/Misc/fp-convert +3.03% MultiSource/Benchmarks/ASC_Sequoia/CrystalMk/CrystalMk -11.20% MultiSource/Benchmarks/Olden/perimeter/perimeter -10.43% and a couple of smaller changes. For perimeter it also results 2.6% a smaller binary. Differential Revision: https://reviews.llvm.org/D30333 llvm-svn: 298799
* [IR] Make SwitchInst::CaseIt almost a normal iterator.Chandler Carruth2017-03-263-16/+24
| | | | | | | | | | | | | | | | | | | | | | | | | This moves it to the iterator facade utilities giving it full random access semantics, etc. It can also now be used with standard algorithms like std::all_of and std::any_of and range adaptors like llvm::reverse. Also make the semantics of iterating match what every other iterator uses and forbid decrementing past the begin iterator. This was used as a hacky way to work around iterator invalidation. However, every instance trying to do this failed to actually avoid touching invalid iterators despite the clear documentation that the removed and all subsequent iterators become invalid including the end iterator. So I've added a return of the next iterator to removeCase and rewritten the loops that were doing this to correctly follow the iterator pattern of either incremneting or removing and assigning fresh values to the iterator and the end. In one case we were trying to go backwards to make this cleaner but it doesn't actually work. I've made that code match the code we use everywhere else to remove cases as we iterate. This changes the order of cases in one test output and I moved that test to CHECK-DAG so it wouldn't care -- the order isn't semantically meaningful anyways. llvm-svn: 298791
* [InstCombine] Change the interface of SimplifyDemandedBits so that it takes ↵Craig Topper2017-03-253-46/+46
| | | | | | | | the instruction and operand instead of the Use. The first thing it did was get the User for the Use to get the instruction back. This requires looking through the Uses for the User using the waymarking walk. That's pretty fast, but its probably still better to just pass the Instruction we already had. llvm-svn: 298772
* [NewGVN] Adjust NDEBUG markers.Davide Italiano2017-03-251-2/+2
| | | | | | | This avoids 'used but not defined' warnings in Release builds with GCC. llvm-svn: 298760
* [asan] Put ctor/dtor in comdat.Evgeniy Stepanov2017-03-251-9/+41
| | | | | | | | | | | | | | | | When possible, put ASan ctor/dtor in comdat. The only reason not to is global registration, which can be TU-specific. This is not the case when there are no instrumented globals. This is also limited to ELF targets, because MachO does not have comdat, and COFF linkers may GC comdat constructors. The benefit of this is a lot less __asan_init() calls: one per DSO instead of one per TU. It's also necessary for the upcoming gc-sections-for-globals change on Linux, where multiple references to section start symbols trigger quadratic behaviour in gold linker. llvm-svn: 298756
* Revert r298711 "[InstCombine] Provide a way to calculate KnownZero/One for ↵Craig Topper2017-03-241-5/+4
| | | | | | | | Add/Sub in SimplifyDemandedUseBits without recursing into ComputeKnownBits" Tsan bot is failing. llvm-svn: 298745
* Revert r298620: [LV] Vectorize GEPsIvan Krasin2017-03-241-117/+67
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Reason: breaks linking Chromium with LLD + ThinLTO (a pass crashes) LLVM bug: https://bugs.llvm.org//show_bug.cgi?id=32413 Original change description: [LV] Vectorize GEPs This patch adds support for vectorizing GEPs. Previously, we only generated vector GEPs on-demand when creating gather or scatter operations. All GEPs from the original loop were scalarized by default, and if a pointer was to be stored to memory, we would have to build up the pointer vector with insertelement instructions. With this patch, we will vectorize all GEPs that haven't already been marked for scalarization. The patch refines collectLoopScalars to more exactly identify the scalar GEPs. The function now more closely resembles collectLoopUniforms. And the patch moves vector GEP creation out of vectorizeMemoryInstruction and into the main vectorization loop. The vector GEPs needed for gather and scatter operations will have already been generated before vectoring the memory accesses. Original Differential Revision: https://reviews.llvm.org/D30710 llvm-svn: 298735
* [asan] Delay creation of asan ctor.Evgeniy Stepanov2017-03-242-21/+26
| | | | | | | | | Create the constructor in the module pass. This in needed for the GC-friendly globals change, where the constructor can be put in a comdat in some cases, but we don't know about that in the function pass. llvm-svn: 298731
* AMDGPU: Fold rcp/rsq of undef to undefMatt Arsenault2017-03-241-2/+15
| | | | llvm-svn: 298725
* TTI: Split IsSimple in MemIntrinsicInfoMatt Arsenault2017-03-241-13/+9
| | | | | | All this did before was assert in EarlyCSE. llvm-svn: 298724
* [ThinLTO] Correct counting of functions in inliner statsTeresa Johnson2017-03-241-0/+2
| | | | | | | | | | | | Summary: Declarations need to be filtered out when counting functions. Reviewers: eraman Subscribers: Prazek, llvm-commits Differential Revision: https://reviews.llvm.org/D31336 llvm-svn: 298720
* [InstCombine] Provide a way to calculate KnownZero/One for Add/Sub in ↵Craig Topper2017-03-241-4/+5
| | | | | | | | SimplifyDemandedUseBits without recursing into ComputeKnownBits SimplifyDemandedUseBits for Add/Sub already recursed down LHS and RHS for simplifying bits. If that didn't provide any simplifications we fall back to calling computeKnownBits which will recurse again. Instead just take the known bits for LHS and RHS we already have and call into a new function in ValueTracking that can calculate the known bits given the LHS/RHS bits. llvm-svn: 298711
* Make GCC happy again.Benjamin Kramer2017-03-241-2/+1
| | | | llvm-svn: 298702
OpenPOWER on IntegriCloud