summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
* [RS4GC] Use "deopt" operand bundlesSanjoy Das2015-10-161-36/+175
| | | | | | | | | | | | | | | | | | | | | | Summary: This is a step towards using operand bundles to carry deopt state till RewriteStatepointsForGC. The change adds a flag to RewriteStatepointsForGC that teaches it to pick up deopt state from a `"deopt"` operand bundle attached to the `call` or `invoke` it is wrapping. The command line flag added, `-rs4gc-use-deopt-bundles`, will only exist for a short while. Once we are able to pipe deopt bundle state through the full optimization pipeline without problems, we will "constant fold" `-rs4gc-use-deopt-bundles` to `true`. Reviewers: swaroop.sridhar, reames Subscribers: llvm-commits, sanjoy Differential Revision: http://reviews.llvm.org/D13372 llvm-svn: 250489
* [IndVars] Rename getExtend; NFCSanjoy Das2015-10-161-17/+17
| | | | | | | Rename `IndVarSimplify::getExtend` to `IndVarSimplify::createExtendInst` to make it obvious that it creates `llvm::Instruction` s. llvm-svn: 250484
* [IndVars] Have `cloneArithmeticIVUser` guess betterSanjoy Das2015-10-161-12/+74
| | | | | | | | | | | | | | | | | | | | | Summary: `cloneArithmeticIVUser` currently trips over expression like `add %iv, -1` when `%iv` is being zero extended -- it tries to construct the widened use as `add %iv.zext, zext(-1)` and (correctly) fails to prove equivalence to `zext(add %iv, -1)` (here the SCEV for `%iv` is `{1,+,1}`). This change teaches `IndVars` to try sign extending the non-IV operand if that makes the newly constructed IV use equivalent to the widened narrow IV use. Reviewers: atrick, hfinkel, reames Subscribers: sanjoy, llvm-commits Differential Revision: http://reviews.llvm.org/D13717 llvm-svn: 250483
* [IndVars] Extract out a few local variables; NFCSanjoy Das2015-10-161-24/+32
| | | | llvm-svn: 250482
* [IndVars] Split `WidenIV::cloneIVUser`; NFCSanjoy Das2015-10-161-25/+71
| | | | | | | | | | | | | | | Summary: This NFC splitting is intended to make a later diff easier to follow. It just tail duplicates `cloneIVUser` into `cloneArithmeticIVUser` and `cloneBitwiseIVUser`. Reviewers: atrick, hfinkel, reames Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D13716 llvm-svn: 250481
* Revert "[safestack] Fast access to the unsafe stack pointer on AArch64/Android."Evgeniy Stepanov2015-10-151-1/+56
| | | | | | Breaks the hexagon buildbot. llvm-svn: 250461
* [safestack] Fast access to the unsafe stack pointer on AArch64/Android.Evgeniy Stepanov2015-10-151-56/+1
| | | | | | | | | | | | | | | | Android libc provides a fixed TLS slot for the unsafe stack pointer, and this change implements direct access to that slot on AArch64 via __builtin_thread_pointer() + offset. This change also moves more code into TargetLowering and its target-specific subclasses to get rid of target-specific codegen in SafeStackPass. This change does not touch the ARM backend because ARM lowers builting_thread_pointer as aeabi_read_tp, which is not available on Android. llvm-svn: 250456
* Revert 250343 and 250344Philip Reames2015-10-151-117/+0
| | | | | | | | | | | | | | | | | | | | | | | | Turns out this approach is buggy. In discussion about follow on work, Sanjoy pointed out that we could be subject to circular logic problems. Consider: if (i u< L) leave() if ((i + 1) u< L) leave() print(a[i] + a[i+1]) If we know that L is less than UINT_MAX, we could possible prove (in a control dependent way) that i + 1 does not overflow. This gives us: if (i u< L) leave() if ((i +nuw 1) u< L) leave() print(a[i] + a[i+1]) If we now do the transform this patch proposed, we end up with: if ((i +nuw 1) u< L) leave_appropriately() print(a[i] + a[i+1]) That would be a miscompile when i==-1. The problem here is that the control dependent nuw bits got used to prove something about the first condition. That's obviously invalid. This won't happen today, but since I plan to enhance LVI/CVP with exactly that transform at some point in the not too distant future... llvm-svn: 250430
* Sample Profiles - Adjust integer types. Mostly NFC.Diego Novillo2015-10-151-22/+29
| | | | | | | | | | | | | | This adjusts all integers in the reader/writer to reflect the types stored on profile files. They should all be unsigned 32-bit or 64-bit values. Changed all associated internal types to be uint32_t or uint64_t. The only place that needed some adjustments is in the sample profile transformation. Altough the weight read from the profile are 64-bit values, the internal API for branch weights only accepts 32-bit values. The pass now saturates weights that overflow uint32_t. llvm-svn: 250427
* [ScalarOpts] Remove dead code.Benjamin Kramer2015-10-155-53/+9
| | | | | | Does not touch debug dumpers. NFC. llvm-svn: 250417
* Recommit r250345, it was reverted in r250366 to investigate a bot failure.Manman Ren2015-10-151-5/+116
| | | | | | Our internal bot is still red after r250366. llvm-svn: 250415
* Temporarily revert r250345 to sort out bot failure.Manman Ren2015-10-151-116/+5
| | | | | | | | | | | | | | With r250345 and r250343, we start to observe the following failure when bootstrap clang with lto and pgo: PHI node entries do not match predecessors! %.sroa.029.3.i = phi %"class.llvm::SDNode.13298"* [ null, %30953 ], [ null, %31017 ], [ null, %30998 ], [ null, %_ZN4llvm8dyn_castINS_14ConstantSDNodeENS_7SDValueEEENS_10cast_rettyIT_T0_E8ret_typeERS5_.exit.i.1804 ], [ null, %30975 ], [ null, %30991 ], [ null, %_ZNK4llvm3EVT13getScalarTypeEv.exit.i.1812 ], [ %..sroa.029.0.i, %_ZN4llvm11SmallVectorIiLj8EED1Ev.exit.i.1826 ], !dbg !451895 label %30998 label %_ZNK4llvm3EVTeqES0_.exit19.thread.i LLVM ERROR: Broken function found, compilation aborted! I will re-commit this if the bot does not recover. llvm-svn: 250366
* Update the branch weight metadata in JumpThreading pass.Cong Hou2015-10-141-5/+116
| | | | | | | | | | Currently in JumpThreading pass, the branch weight metadata is not updated after CFG modification. Consider the jump threading on PredBB, BB, and SuccBB. After jump threading, the weight on BB->SuccBB should be adjusted as some of it is contributed by the edge PredBB->BB, which doesn't exist anymore. This patch tries to update the edge weight in metadata on BB->SuccBB by scaling it by 1 - Freq(PredBB->BB) / Freq(BB->SuccBB). This is the third attempt to submit this patch, while the first two led to failures in some FDO tests. After investigation, it is the edge weight normalization that caused those failures. In this patch the edge weight normalization is fixed so that there is no zero weight in the output and the sum of all weights can fit in 32-bit integer. Several unit tests are added. Differential revision: http://reviews.llvm.org/D10979 llvm-svn: 250345
* [SimplifyCFG] Speculatively flatten CFG based on profiling metadataPhilip Reames2015-10-141-7/+124
| | | | | | | | | | If we have a series of branches which are all unlikely to fail, we can possibly combine them into a single check on the fastpath combined with a bit of dispatch logic on the slowpath. We don't want to do this unconditionally since it requires speculating instructions past a branch, but if the profiling metadata on the branch indicates profitability, this can reduce the number of checks needed along the fast path. The canonical example this is trying to handle is removing the second bounds check implied by the Java code: a[i] + a[i+1]. Note that it can currently only do so for really simple conditions and the values of a[i] can't be used anywhere except in the addition. (i.e. the load has to have been sunk already and not prevent speculation.) I plan on extending this transform over the next few days to handle alternate sequences. Differential Revision: http://reviews.llvm.org/D13070 llvm-svn: 250343
* [LoopUnswitch] Correct misleading comments.Chen Li2015-10-141-2/+1
| | | | | | | | | | Reviewers: reames Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D13738 llvm-svn: 250317
* Revert r250204 and r250240 due to bot failure. We failed to build PGO-ed clang.Manman Ren2015-10-141-118/+5
| | | | llvm-svn: 250264
* [msan] Fix crash on multiplication by a non-integer constant.Evgeniy Stepanov2015-10-141-9/+15
| | | | | | Fixes PR25160. llvm-svn: 250260
* [InlineFunction] Correctly inline TerminatePadInstDavid Majnemer2015-10-131-5/+10
| | | | | | | | | | | We forgot to append the terminatepad's arguments which resulted in us treating the old terminatepad as an argument to the new terminatepad causing us to crash immediately. Instead, add the old terminatepad's arguments to the new terminatepad. This fixes PR25155. llvm-svn: 250234
* Typo.Chad Rosier2015-10-131-1/+1
| | | | llvm-svn: 250224
* Scalar: Remove remaining ilist iterator implicit conversionsDuncan P. N. Exon Smith2015-10-1327-240/+240
| | | | | | | | | | | | | | | | | | | Remove remaining `ilist_iterator` implicit conversions from LLVMScalarOpts. This change exposed some scary behaviour in lib/Transforms/Scalar/SCCP.cpp around line 1770. This patch changes a call from `Function::begin()` to `&Function::front()`, since the return was immediately being passed into another function that takes a `Function*`. `Function::front()` started to assert, since the function was empty. Note that `Function::end()` does not point at a legal `Function*` -- it points at an `ilist_half_node` -- so the other function was getting garbage before. (I added the missing check for `Function::isDeclaration()`.) Otherwise, no functionality change intended. llvm-svn: 250211
* Update the branch weight metadata in JumpThreading pass.Cong Hou2015-10-131-5/+118
| | | | | | | | Currently in JumpThreading pass, the branch weight metadata is not updated after CFG modification. Consider the jump threading on PredBB, BB, and SuccBB. After jump threading, the weight on BB->SuccBB should be adjusted as some of it is contributed by the edge PredBB->BB, which doesn't exist anymore. This patch tries to update the edge weight in metadata on BB->SuccBB by scaling it by 1 - Freq(PredBB->BB) / Freq(BB->SuccBB). Differential revision: http://reviews.llvm.org/D10979 llvm-svn: 250204
* [PGO]: Eliminate calls to __llvm_profile_register_function for Linux.Xinliang David Li2015-10-131-0/+3
| | | | | | | | | On Linux, the profile runtime can use __start_SECTNAME and __stop_SECTNAME symbols defined by the linker to locate the start and end location of a named section (with C name). This eliminates the need for instrumented binary to call __llvm_profile_register_function during start-up time. llvm-svn: 250199
* Scalar: Remove some implicit ilist iterator conversions, NFCDuncan P. N. Exon Smith2015-10-1310-61/+62
| | | | | | | Remove some of the implicit ilist iterator conversions in LLVMScalarOpts. More to go. llvm-svn: 250197
* IPO: Remove implicit ilist iterator conversions, NFCDuncan P. N. Exon Smith2015-10-1313-99/+97
| | | | llvm-svn: 250187
* Instrumentation: Remove ilist iterator implicit conversions, NFCDuncan P. N. Exon Smith2015-10-139-57/+59
| | | | llvm-svn: 250186
* InstCombine: Remove ilist iterator implicit conversions, NFCDuncan P. N. Exon Smith2015-10-138-37/+38
| | | | | | | Stop relying on implicit conversions of ilist iterators in LLVMInstCombine. No functionality change intended. llvm-svn: 250183
* [InstCombine][SSE4A] Remove broken INSERTQI range combining optimizationSimon Pilgrim2015-10-131-45/+4
| | | | | | | | As discussed in D13348 - the INSERTQI range combining code is wrong in that it confuses the insertion bit index with an extraction bit index. The remaining legal combines are very unlikely (especially once we've converted to shuffles in D13348) so I'm removing the optimization. llvm-svn: 250160
* [GlobalsAA] Turn GlobalsAA on again by defaultJames Molloy2015-10-131-1/+1
| | | | | | | | Now that all the known faults with GlobalsAA have been fixed, flip the big switch on -enable-non-lto-gmr again. Feel free to pester me with any more bugs found, and don't hesitate to flip the switch back off. llvm-svn: 250157
* [IndVars] NFC Cleanup.Sanjoy Das2015-10-131-66/+62
| | | | | | | | - Rename methods according to the LLVM Coding Style - Merge adjacent anonymous namespace block - Use `auto` in two places llvm-svn: 250152
* Revert 250089 due to bot failure. It failed when building clang itself with PGO.Manman Ren2015-10-131-118/+5
| | | | llvm-svn: 250145
* TransformUtils: Remove implicit ilist iterator conversions, NFCDuncan P. N. Exon Smith2015-10-1325-254/+253
| | | | | | | | | | | Continuing the work from last week to remove implicit ilist iterator conversions. First related commit was probably r249767, with some more motivation in r249925. This edition gets LLVMTransformUtils compiling without the implicit conversions. No functional change intended. llvm-svn: 250142
* Update the branch weight metadata in JumpThreading pass.Cong Hou2015-10-121-5/+118
| | | | | | | | In JumpThreading pass, the branch weight metadata is not updated after CFG modification. Consider the jump threading on PredBB, BB, and SuccBB. After jump threading, the weight on BB->SuccBB should be adjusted as some of it is contributed by the edge PredBB->BB, which doesn't exist anymore. This patch tries to update the edge weight in metadata on BB->SuccBB by scaling it by 1 - Freq(PredBB->BB) / Freq(BB->SuccBB). Differential revision: http://reviews.llvm.org/D10979 llvm-svn: 250089
* GlobalOpt does not treat externally_initialized globals correctlyOliver Stannard2015-10-122-1/+5
| | | | | | | | GlobalOpt currently merges stores into the initialisers of internal, externally_initialized globals, but should not do so as the value of the global may change between the initialiser and any code in the module being run. llvm-svn: 250035
* [LoopVectorize] Shrink integer operations into the smallest type possibleJames Molloy2015-10-121-11/+180
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | C semantics force sub-int-sized values (e.g. i8, i16) to be promoted to int type (e.g. i32) whenever arithmetic is performed on them. For targets with native i8 or i16 operations, usually InstCombine can shrink the arithmetic type down again. However InstCombine refuses to create illegal types, so for targets without i8 or i16 registers, the lengthening and shrinking remains. Most SIMD ISAs (e.g. NEON) however support vectors of i8 or i16 even when their scalar equivalents do not, so during vectorization it is important to remove these lengthens and truncates when deciding the profitability of vectorization. The algorithm this uses starts at truncs and icmps, trawling their use-def chains until they terminate or instructions outside the loop are found (or unsafe instructions like inttoptr casts are found). If the use-def chains starting from different root instructions (truncs/icmps) meet, they are unioned. The demanded bits of each node in the graph are ORed together to form an overall mask of the demanded bits in the entire graph. The minimum bitwidth that graph can be truncated to is the bitwidth minus the number of leading zeroes in the overall mask. The intention is that this algorithm should "first do no harm", so it will never insert extra cast instructions. This is why the use-def graphs are unioned, so that subgraphs with different minimum bitwidths do not need casts inserted between them. This algorithm works hard to reduce compile time impact. DemandedBits are only queried if there are extends of illegal types and if a truncate to an illegal type is seen. In the general case, this results in a simple linear scan of the instructions in the loop. No non-noise compile time impact was seen on a clang bootstrap build. llvm-svn: 250032
* [InstCombine][X86][XOP] Combine XOP integer vector comparisons to native IRSimon Pilgrim2015-10-111-0/+53
| | | | | | We now have lowering support for XOP PCOM/PCOMU instructions. llvm-svn: 249977
* [IndVars] Use `auto`; NFCSanjoy Das2015-10-101-6/+4
| | | | llvm-svn: 249944
* Generalize convergent check to handle invokes as well as calls.Owen Anderson2015-10-091-4/+4
| | | | llvm-svn: 249892
* Teach LoopUnswitch not to perform non-trivial unswitching on loops ↵Owen Anderson2015-10-091-0/+14
| | | | | | | | | | | containing convergent operations. Doing so could cause the post-unswitching convergent ops to be control-dependent on the unswitch condition where they were not before. This check could be refined to allow unswitching where the convergent operation was already control-dependent on the unswitch condition. llvm-svn: 249874
* Refine the definition of convergent to only disallow the addition of new ↵Owen Anderson2015-10-091-1/+2
| | | | | | | | | | control dependencies. This covers the common case of operations that cannot be sunk. Operations that cannot be hoisted should already be handled properly via the safe-to-speculate rules and mechanisms. llvm-svn: 249865
* Make HeaderLineno a local variable.Dehao Chen2015-10-091-12/+8
| | | | | | | | http://reviews.llvm.org/D13576 As we are using hierarchical profile, there is no need to keep HeaderLineno a member variable. This is because each level of the inline stack will have its own header lineno. One should use the head lineno of its own inline stack level instead of the actual symbol. llvm-svn: 249848
* [MemCpyOpt] Fix wrong merging adjacent nontemporal stores into memset calls.Andrea Di Biagio2015-10-091-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Pass MemCpyOpt doesn't check if a store instruction is nontemporal. As a consequence, adjacent nontemporal stores are always merged into a memset call. Example: ;;; define void @foo(<4 x float>* nocapture %p) { entry: store <4 x float> zeroinitializer, <4 x float>* %p, align 16, !nontemporal !0 %p1 = getelementptr inbounds <4 x float>, <4 x float>* %dst, i64 1 store <4 x float> zeroinitializer, <4 x float>* %p1, align 16, !nontemporal !0 ret void } !0 = !{i32 1} ;;; In this example, the two nontemporal stores are combined to a memset of zero which does not preserve the nontemporal hint. Later on the backend (tested on a x86-64 corei7) expands that memset call into a sequence of two normal 16-byte aligned vector stores. opt -memcpyopt example.ll -S -o - | llc -mcpu=corei7 -o - Before: xorps %xmm0, %xmm0 movaps %xmm0, 16(%rdi) movaps %xmm0, (%rdi) With this patch, we no longer merge nontemporal stores into calls to memset. In this example, llc correctly expands the two stores into two movntps: xorps %xmm0, %xmm0 movntps %xmm0, 16(%rdi) movntps %xmm0, (%rdi) In theory, we could extend the usage of !nontemporal metadata to memcpy/memset calls. However a change like that would only have the effect of forcing the backend to expand !nontemporal memsets back to sequences of store instructions. A memset library call would not have exactly the same semantic of a builtin !nontemporal memset call. So, SelectionDAG will have to conservatively expand it back to a sequence of !nontemporal stores (effectively undoing the merging). Differential Revision: http://reviews.llvm.org/D13519 llvm-svn: 249820
* [EarlyCSE] Address post commit review for r249523.Arnaud A. de Grandmaison2015-10-091-10/+10
| | | | llvm-svn: 249814
* [RS4GC] Refactoring to make a later change easier, NFCISanjoy Das2015-10-081-19/+22
| | | | | | | | | | | | | | Summary: These non-semantic changes will help make a later change adding support for deopt operand bundles more streamlined. Reviewers: reames, swaroop.sridhar Subscribers: sanjoy, llvm-commits Differential Revision: http://reviews.llvm.org/D13491 llvm-svn: 249779
* [PlaceSafeopints] Extract out `callsGCLeafFunction`, NFCSanjoy Das2015-10-082-28/+18
| | | | | | | | | | | | | Summary: This will be used in a later change to RewriteStatepointsForGC. Reviewers: reames, swaroop.sridhar Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D13490 llvm-svn: 249777
* [RS4GC] Don't copy ADT's unneccessarily, NFCISanjoy Das2015-10-081-3/+3
| | | | | | | | | | | | Summary: Use `const auto &` instead of `auto` in `makeStatepointExplicit`. Reviewers: reames, swaroop.sridhar Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D13454 llvm-svn: 249776
* New MSan mapping layout (llvm part).Evgeniy Stepanov2015-10-081-7/+15
| | | | | | | | | | | | | | | | | | This is an implementation of https://github.com/google/sanitizers/issues/579 It has a number of advantages over the current mapping: * Works for non-PIE executables. * Does not require ASLR; as a consequence, debugging MSan programs in gdb no longer requires "set disable-randomization off". * Supports linux kernels >=4.1.2. * The code is marginally faster and smaller. This is an ABI break. We never really promised ABI stability, but this patch includes a courtesy escape hatch: a compile-time macro that reverts back to the old mapping layout. llvm-svn: 249753
* Add Triple::isAndroid().Evgeniy Stepanov2015-10-082-2/+2
| | | | | | | This is a simple refactoring that replaces Triple.getEnvironment() checks for Android with Triple.isAndroid(). llvm-svn: 249750
* [InstCombine] transform masking off of an FP sign bit into a fabs() ↵Sanjay Patel2015-10-081-4/+19
| | | | | | | | | | | | | | | | | | intrinsic call (PR24886) This is a partial fix for PR24886: https://llvm.org/bugs/show_bug.cgi?id=24886 Without this IR transform, the backend (x86 at least) was producing inefficient code. This patch is making 2 assumptions: 1. The canonical form of a fabs() operation is, in fact, the LLVM fabs() intrinsic. 2. The high bit of an FP value is always the sign bit; as noted in the bug report, this isn't specified by the LangRef. Differential Revision: http://reviews.llvm.org/D13076 llvm-svn: 249702
* [RS4GC] Use AssertingVH for RematerializedValueMapTy, NFCISanjoy Das2015-10-071-1/+2
| | | | | | | | | | Reviewers: reames, swaroop.sridhar Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D13489 llvm-svn: 249620
* [IndVars] Preserve LCSSA in `eliminateIdentitySCEV`Sanjoy Das2015-10-071-0/+3
| | | | | | | | | | | | | | | | | | | | Summary: After r249211, SCEV can see through some LCSSA phis. Add a `replacementPreservesLCSSAForm` check before replacing uses of these phi nodes with a simplified use of the induction variable to avoid breaking LCSSA. Fixes 25047. Depends on D13460. Reviewers: atrick, hfinkel Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D13461 llvm-svn: 249575
OpenPOWER on IntegriCloud