summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
* Revert "Invoke salvageDebugInfo from CodeGenPrepare's SinkCast()"Adrian Prantl2017-11-031-1/+1
| | | | | | This reverts commit 317342 while investigating bot breakage. llvm-svn: 317345
* Invoke salvageDebugInfo from CodeGenPrepare's SinkCast()Adrian Prantl2017-11-031-1/+1
| | | | | | | | This preserves the debug info for the cast operation in the original location. rdar://problem/33460652 llvm-svn: 317340
* [LICM] sink through non-trivially replicable PHIJun Bum Lim2017-11-031-56/+140
| | | | | | | | | | | | | | | | | | | Summary: The current LICM allows sinking an instruction only when it is exposed to exit blocks through a trivially replacable PHI of which all incoming values are the same instruction. This change enhance LICM to sink a sinkable instruction through non-trivially replacable PHIs by spliting predecessors of loop exits. Reviewers: hfinkel, majnemer, davidxl, bmakam, mcrosier, danielcdh, efriedma, jtony Reviewed By: efriedma Subscribers: nemanjai, dberlin, llvm-commits Differential Revision: https://reviews.llvm.org/D37163 llvm-svn: 317335
* [LoopPredication] NFC: Refactored code to separate out functions being reusedAnna Thomas2017-11-031-62/+92
| | | | | | | | | | | | Summary: Refactored the code to separate out common functions that are being reused. This is to reduce the changes for changes coming up wrt loop predication with reverse loops. This refactoring is what we have in our downstream code. llvm-svn: 317324
* [ADCE] Use MapVector for BlockInfo to make iteration order deterministicMikael Holmen2017-11-031-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: Also added a reserve() method to MapVector since we want to use that from ADCE. DenseMap does not provide deterministic iteration order so with that we will handle the members of BlockInfo in random order, eventually leading to random order of the blocks in the predecessor lists. Without this change, I get the same predecessor order in about 90% of the time when I compile a certain reproducer and in 10% I get a different one. No idea how to make a proper test case for this. Reviewers: kuhar, david2050 Reviewed By: kuhar Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D39593 llvm-svn: 317323
* [PartialInliner] Skip call sites where inlining fails.Florian Hahn2017-11-031-7/+9
| | | | | | | | | | | | | | | | | | Summary: InlineFunction can fail, for example when trying to inline vararg fuctions. In those cases, we do not want to bump partial inlining counters or set AnyInlined to true, because this could leave an unused function hanging around. Reviewers: davidxl, davide, gyiu Reviewed By: davide Subscribers: llvm-commits, eraman Differential Revision: https://reviews.llvm.org/D39581 llvm-svn: 317314
* [LSR] Clarify a comment. NFC.Vedant Kumar2017-11-031-1/+1
| | | | llvm-svn: 317295
* IndVarSimplify: preserve debug information attached to widened PHI nodes.Adrian Prantl2017-11-021-0/+10
| | | | | | | | | | This fixes PR35015. https://bugs.llvm.org/show_bug.cgi?id=35015 Differential Revision: https://reviews.llvm.org/D39345 llvm-svn: 317282
* Irreducible loop metadata for more accurate block frequency under PGO.Hiroshi Yamauchi2017-11-021-2/+26
| | | | | | | | | | | | | | | | | | | | | | | Summary: Currently the block frequency analysis is an approximation for irreducible loops. The new irreducible loop metadata is used to annotate the irreducible loop headers with their header weights based on the PGO profile (currently this is approximated to be evenly weighted) and to help improve the accuracy of the block frequency analysis for irreducible loops. This patch is a basic support for this. Reviewers: davidxl Reviewed By: davidxl Subscribers: mehdi_amini, llvm-commits, eraman Differential Revision: https://reviews.llvm.org/D39028 llvm-svn: 317278
* [LoopPredication] Enable predication when latchCheckIV is wider than rangeCheckAnna Thomas2017-11-021-10/+96
| | | | | | | | | | | | | | | | | | | | | | Summary: This patch allows us to predicate range checks that have a type narrower than the latch check type. We leverage SCEV analysis to identify a truncate for the latchLimit and latchStart. There is also safety checks in place which requires the start and limit to be known at compile time. We require this to make sure that the SCEV truncate expr for the IV corresponding to the latch does not cause us to lose information about the IV range. Added tests show the loop predication over range checks that are of various types and are narrower than the latch type. This enhancement has been in our downstream tree for a while. Reviewers: apilipenko, sanjoy, mkazantsev Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D39500 llvm-svn: 317269
* Strip off invariant.start because memory locations arent invariantAnna Thomas2017-11-021-9/+33
| | | | | | | | | | | | The original change was reverted in rL317217 because of the failure in the RS4GC testcase. I couldn't reproduce the failure on my local machine (macbook) but could reproduce it on a linux box. The failure was around removing the uses of invariant.start. The fix here is to just RAUW undef (which was the first implementation in D39388). This is perfectly valid IR as discussed in the review. llvm-svn: 317225
* Revert "[RS4GC] Strip off invariant.start because memory locations arent ↵Anna Thomas2017-11-021-39/+9
| | | | | | | | invariant" This reverts commit r317215, investigating the test failure. llvm-svn: 317217
* [RS4GC] Strip off invariant.start because memory locations arent invariantAnna Thomas2017-11-021-9/+39
| | | | | | | | | | | | | | | | | | | | Summary: Invariant.start on memory locations has the property that the memory location is unchanging. However, this is not true in the face of rewriting statepoints for GC. Teach RS4GC about removing invariant.start so that optimizations after RS4GC does not incorrect sink a load from the memory location past a statepoint. Added test showcasing the issue. Reviewers: reames, apilipenko, dneilson Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D39388 llvm-svn: 317215
* Revert "[ExpandMemCmp] Split ExpandMemCmp from CodeGen into its own pass."Clement Courbet2017-11-023-830/+0
| | | | | | | | | undefined reference to `llvm::TargetPassConfig::ID' on clang-ppc64le-linux-multistage This reverts commit eea333c33fa73ad225ef28607795984829f65688. llvm-svn: 317213
* [ExpandMemCmp] Split ExpandMemCmp from CodeGen into its own pass.Clement Courbet2017-11-023-0/+830
| | | | | | | | | | | | | | | | | Summary: This is mostly a noop (most of the test diffs are renamed blocks). There are a few temporary register renames (eax<->ecx) and a few blocks are shuffled around. See the discussion in PR33325 for more details. Reviewers: spatel Subscribers: mgorny Differential Revision: https://reviews.llvm.org/D39456 llvm-svn: 317211
* [SimplifyCFG] Discard speculated dbg intrinsicsBjorn Pettersson2017-11-021-1/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | Summary: SpeculativelyExecuteBB can flatten the CFG by doing speculative execution followed by a select instruction. When the speculatively executed BB contained dbg intrinsics the result could be a little bit weird, since those dbg intrinsics were inserted before the select in the flattened CFG. So when single stepping in the debugger, printing the value of the variable referenced in the dbg intrinsic, it could happen that it looked like the variable had values that never actually were assigned to the variable. This patch simply discards all dbg intrinsics that were found in the speculatively executed BB. Reviewers: aprantl, chandlerc, craig.topper Reviewed By: aprantl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D39494 llvm-svn: 317198
* loop-unroll: teach remapInstruction to update dbg.value intrinsics.Adrian Prantl2017-11-011-1/+15
| | | | | | | | Fixes PR35112. https://bugs.llvm.org/show_bug.cgi?id=35112 llvm-svn: 317138
* loop-rotate: avoid duplicating dbg.value intrinsics in the entry block.Adrian Prantl2017-11-011-0/+24
| | | | | | | | This fixes the second half of PR35113. This reapplies r317106 without modifications. llvm-svn: 317121
* loop-rotate: eliminate duplicate debug intrinsics after splicing.Adrian Prantl2017-11-011-1/+26
| | | | | | | | | Fixes part of PR35113. This reapplies r317105 with an additional check for isa<Instruction> as found by the bots. llvm-svn: 317120
* Include GUIDs from the same module when computing GUIDs that needs to be ↵Dehao Chen2017-11-011-15/+16
| | | | | | | | | | | | | | | | imported. Summary: In the compile phase of SamplePGO+ThinLTO, ICP is not invoked. Instead, indirect call targets will be included as function metadata for ThinIndex to buidl the call graph. This should not only include functions defined in other modules, but also functions defined in the same module, otherwise ThinIndex may find the callee dead and eliminate it, while ICP in backend will revive the symbol, which leads to undefined symbol. Reviewers: tejohnson Reviewed By: tejohnson Subscribers: sanjoy, llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D39480 llvm-svn: 317118
* Revert 317016 and 317048Philip Reames2017-11-011-44/+50
| | | | | | The former appears to have introduced a miscompile in a stage2 clang build. Revert so I can investigate offline. llvm-svn: 317116
* Revert r317105 to investigate bot breakage.Adrian Prantl2017-11-011-23/+1
| | | | llvm-svn: 317110
* Revert r317106 to facilitate reverting r317105.Adrian Prantl2017-11-011-24/+0
| | | | llvm-svn: 317109
* LTO: Apply global DCE to ThinLTO modules at LTO opt level 0.Peter Collingbourne2017-11-011-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | This is necessary because DCE is applied to full LTO modules. Without this change, a reference from a dead ThinLTO global to a dead full LTO global will result in an undefined reference at link time. This problem is only observable when --gc-sections is disabled, or when targeting COFF, as the COFF port of lld requires all symbols to have a definition even if all references are dead (this is consistent with link.exe). This change also adds an EliminateAvailableExternally pass at -O0. This is necessary to handle the situation on Windows where a non-prevailing copy of a linkonce_odr function has an SEH filter function; any such filters must be DCE'd because they will contain a call to the llvm.localrecover intrinsic, passing as an argument the address of the function that the filter belongs to, and llvm.localrecover requires this function to be defined locally. Fixes PR35142. Differential Revision: https://reviews.llvm.org/D39484 llvm-svn: 317108
* loop-rotate: avoid duplicating dbg.value intrinsics in the entry block.Adrian Prantl2017-11-011-0/+24
| | | | | | This fixes the second half of PR35113. llvm-svn: 317106
* loop-rotate: eliminate duplicate debug intrinsics after splicing.Adrian Prantl2017-11-011-1/+23
| | | | | | Fixes part of PR35113. llvm-svn: 317105
* Revert rL311205 "[IRCE] Fix buggy behavior in Clamp"Max Kazantsev2017-11-011-2/+1
| | | | | | | | | | | | | This patch reverts rL311205 that was initially a wrong fix. The real problem was in intersection of signed and unsigned ranges (see rL316552), and the patch being reverted masked the problem instead of fixing it. By now, the test against which rL311205 was made works OK even without this code. This revert patch also contains a test case that demonstrates incorrect behavior caused by rL311205: it is caused by incorrect choise of signed max instead of unsigned. llvm-svn: 317088
* [CodeExtractor] Fix iterator invalidation in findOrCreateBlockForHoisting.Florian Hahn2017-11-011-1/+3
| | | | | | | | | | | | | | | | | | | Summary: By replacing branches to CommonExitBlock, we remove the node from CommonExitBlock's predecessors, invalidating the iterator. The problem is exposed when the common exit block has multiple predecessors and needs to sink lifetime info. The modification in the test case trigger the issue. Reviewers: davidxl, davide, wmi Reviewed By: davidxl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D39112 llvm-svn: 317084
* [SimplifyIndVar] Inline makIVComparisonInvariant to eleminate code ↵Philip Reames2017-10-311-51/+29
| | | | | | | | duplication [NFC] This formulation might be slightly slower since I eagerly compute the cheap replacements. If anyone sees this having a compile time impact, let me know and I'll use lazy population instead. llvm-svn: 317048
* loop-rotate: simplify code by using llvm::findDbgValues(). (NFC)Adrian Prantl2017-10-311-31/+23
| | | | llvm-svn: 317037
* [coro] Make Spill a proper struct instead of deriving from pair.Benjamin Kramer2017-10-311-12/+10
| | | | | | No functionality change. llvm-svn: 317027
* [SimplifyCFG] Use a more generic name for the selects created by ↵Craig Topper2017-10-311-2/+2
| | | | | | | | | | | | SpeculativelyExecuteBB to prevent long names from being created Currently the selects are created with the names of their inputs concatenated together. It's possible to get cases that chain these selects together resulting in long names due to multiple levels of concatenation. Our internal branch of llvm managed to generate names over 100000 characters in length on a particular test due to an extreme compounding of the names. This patch changes the name to a generic name that is not dependent on its inputs. Differential Revision: https://reviews.llvm.org/D39440 llvm-svn: 317024
* [IndVarSimplify] Extract wrapper around SE-.isLoopInvariantPredicate [NFC]Philip Reames2017-10-311-17/+33
| | | | | | This an intermediate state, the next patch will re-inline the markLoopInvariantPredicate function to reduce code duplication. llvm-svn: 317016
* [IndVarSimplify] Simplify code using a dictionaryPhilip Reames2017-10-311-16/+8
| | | | | | Possibly very slightly slower, but this code is not performance critical and the readability benefit alone is huge. llvm-svn: 317012
* [asan] Upgrade private linkage globals to internal linkage on COFFReid Kleckner2017-10-311-2/+7
| | | | | | | COFF comdats require symbol table entries, which means the comdat leader cannot have private linkage. llvm-svn: 317009
* [LoopVectorize] Replace manual VPlan memory management with unique_ptr.Benjamin Kramer2017-10-311-26/+10
| | | | | | No functionality change intended. llvm-svn: 317003
* [InstCombine] Simplify selects that test cmpxchg instructionsMatthew Simpson2017-10-311-0/+76
| | | | | | | | | | If a select instruction tests the returned flag of a cmpxchg instruction and selects between the returned value of the cmpxchg instruction and its compare operand, the result of the select will always be equal to its false value. Differential Revision: https://reviews.llvm.org/D39383 llvm-svn: 316994
* [LoopUnroll] Clean up remarks for unroll remainderDavid Green2017-10-312-31/+35
| | | | | | | | | | | | | | | | The optimisation remarks for loop unrolling with an unrolled remainder looks something like: test.c:7:18: remark: completely unrolled loop with 3 iterations [-Rpass=loop-unroll] C[i] += A[i*N+j]; ^ test.c:6:9: remark: unrolled loop by a factor of 4 with run-time trip count [-Rpass=loop-unroll] for(int j = 0; j < N; j++) ^ This removes the first of the two messages. Differential revision: https://reviews.llvm.org/D38725 llvm-svn: 316986
* [IRCE][NFC] Rename fields of InductiveRangeCheckMax Kazantsev2017-10-311-23/+23
| | | | | | | | | | Rename `Offset`, `Scale`, `Length` into `Begin`, `Step`, `End` respectively to make naming of similar entities for Ranges and Range Checks more consistent. Differential Revision: https://reviews.llvm.org/D39414 llvm-svn: 316979
* [NFC] Get rid of variables used in assert onlyMax Kazantsev2017-10-311-6/+6
| | | | llvm-svn: 316977
* [IndVarSimplify] Simplify code using preheader assumptionPhilip Reames2017-10-311-44/+6
| | | | | | | | As noted in the nice block comment, the previous code didn't actually handle multi-entry loops correctly, it just assumed SCEV didn't analyze such loops. Given SCEV has comments to the contrary, that seems a bit suspect. More importantly, the pass actually requires loopsimplify form which ensures a loop-preheader is available. Remove the excessive generaility and shorten the code greatly. Note that we do successfully analyze many multi-entry loops, but we do so by converting them to single entry loops. See the added test case. llvm-svn: 316976
* Reapply "[GVN] Prevent LoadPRE from hoisting across instructions that don't ↵Max Kazantsev2017-10-311-0/+97
| | | | | | | | | | | | | | | | | | pass control flow to successors" This patch fixes the miscompile that happens when PRE hoists loads across guards and other instructions that don't always pass control flow to their successors. PRE is now prohibited to hoist across such instructions because there is no guarantee that the load standing after such instruction is still valid before such instruction. For example, a load from under a guard may be invalid before the guard in the following case: int array[LEN]; ... guard(0 <= index && index < LEN); use(array[index]); Differential Revision: https://reviews.llvm.org/D37460 llvm-svn: 316975
* [SimplifyIndVar] Extract out invariant expression handlingPhilip Reames2017-10-311-82/+107
| | | | | | | | Previously, the code returned early from the *function* when it couldn't find a free expansion, it should be returning from the *transform*. I don't have a test case, noticed this via inspection. As a follow up, I'm going to revisit the logic in the extract function. I think that essentially the whole helper routine can be replaced with SCEVExpander, but I wanted to do that in a series of separate commits. llvm-svn: 316974
* Undo accidental commitPhilip Reames2017-10-311-237/+82
| | | | | | These files shouldn't have been submitted in 316967 llvm-svn: 316968
* [CGP] Fix crash on i96 bit multiplyPhilip Reames2017-10-301-82/+237
| | | | | | | | Issue found by llvm-isel-fuzzer on OSS fuzz, https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3725 If anyone actually cares about > 64 bit arithmetic, there's a lot more to do in this area. There's a bunch of obviously wrong code in the same function. I don't have the time to fix all of them and am just using this to understand what the workflow for fixing fuzzer cases might look like. llvm-svn: 316967
* InferAddressSpaces: Fix bug about replacing addrspacecastYaxun Liu2017-10-301-0/+5
| | | | | | | | | | | | | | InferAddressSpaces assumes the pointee type of addrspacecast is the same as the operand, which is not always true and causes invalid IR. This bug cause build failure in HCC. This patch fixes that. Differential Revision: https://reviews.llvm.org/D39432 llvm-svn: 316957
* [NewGVN] Stop assuming PHI args ordering when looking at phi-of-ops.Davide Italiano2017-10-301-1/+1
| | | | | | | | It's not guaranteed. There's a bug open to sort them in predecessor order, but it won't happen anytime soon. In the meanwhile, passes will have to do an O(#preds) scan. Such is life. llvm-svn: 316953
* Create instruction classes for identifying any atomicity of memory ↵Daniel Neilson2017-10-302-5/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | intrinsic. (NFC) Summary: For reference, see: http://lists.llvm.org/pipermail/llvm-dev/2017-August/116589.html This patch fleshes out the instruction class hierarchy with respect to atomic and non-atomic memory intrinsics. With this change, the relevant part of the class hierarchy becomes: IntrinsicInst -> MemIntrinsicBase (methods-only class) -> MemIntrinsic (non-atomic intrinsics) -> MemSetInst -> MemTransferInst -> MemCpyInst -> MemMoveInst -> AtomicMemIntrinsic (atomic intrinsics) -> AtomicMemSetInst -> AtomicMemTransferInst -> AtomicMemCpyInst -> AtomicMemMoveInst -> AnyMemIntrinsic (both atomicities) -> AnyMemSetInst -> AnyMemTransferInst -> AnyMemCpyInst -> AnyMemMoveInst This involves some class renaming: ElementUnorderedAtomicMemCpyInst -> AtomicMemCpyInst ElementUnorderedAtomicMemMoveInst -> AtomicMemMoveInst ElementUnorderedAtomicMemSetInst -> AtomicMemSetInst A script for doing this renaming in downstream trees is included below. An example of where the Any* classes should be used in LLVM is when reasoning about the effects of an instruction (ex: aliasing). --- Script for renaming AtomicMem* classes: PREFIXES="[<,([:space:]]" CLASSES="MemIntrinsic|MemTransferInst|MemSetInst|MemMoveInst|MemCpyInst" SUFFIXES="[;)>,[:space:]]" REGEX="(${PREFIXES})ElementUnorderedAtomic(${CLASSES})(${SUFFIXES})" REGEX2="visitElementUnorderedAtomic(${CLASSES})" FILES=$( grep -E "(${REGEX}|${REGEX2})" -r . | tr ':' ' ' | awk '{print $1}' | sort | uniq ) SED_SCRIPT="s~${REGEX}~\1Atomic\2\3~g" SED_SCRIPT2="s~${REGEX2}~visitAtomic\1~g" for f in $FILES; do echo "Processing: $f" sed -i ".bak" -E "${SED_SCRIPT};${SED_SCRIPT2};${EA_SED_SCRIPT};${EA_SED_SCRIPT2}" $f done Reviewers: sanjoy, deadalnix, apilipenko, anna, skatkov, mkazantsev Reviewed By: sanjoy Subscribers: hfinkel, jholewinski, arsenm, sdardis, nhaehnle, JDevlieghere, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D38419 llvm-svn: 316950
* [GVNHoist] Fix non-deterministic sort order of PHIs for identical instructionsMandeep Singh Grang2017-10-301-1/+1
| | | | | | | | | | | | | | Summary: This fixes failure in Transforms/GVNHoist/hoist.ll uncovered by D39245. Reviewers: hiraditya, spop, dberlin Reviewed By: dberlin Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D39410 llvm-svn: 316949
* [CodeGen][ExpandMemcmp] Allow memcmp to expand to vector loads (2).Clement Courbet2017-10-301-2/+1
| | | | | | | | | | | | - Targets that want to support memcmp expansions now return the list of supported load sizes. - Expansion codegen does not assume that all power-of-two load sizes smaller than the max load size are valid. For examples, this is not the case for x86(32bit)+sse2. Fixes PR34887. llvm-svn: 316905
OpenPOWER on IntegriCloud