summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
...
* Templatize parts of VNCoercion, and add constant-only versions of the ↵Daniel Berlin2017-03-202-68/+111
| | | | | | | | | | | | | | | | | | | | | | | functions to be used in NewGVN. NFCI. Summary: This is ground work for the changes to enable coercion in NewGVN. GVN doesn't care if they end up constant because it eliminates as it goes. NewGVN cares. IRBuilder and ConstantFolder deliberately present the same interface, so we use this to our advantage to templatize our functions to make them either constant only or not. Reviewers: davide Subscribers: llvm-commits, Prazek Differential Revision: https://reviews.llvm.org/D30928 llvm-svn: 298262
* Use MutableArrayRef for APFloat::convertToIntegerSimon Pilgrim2017-03-201-2/+3
| | | | | | As discussed on D31074, use MutableArrayRef for destination integer buffers to help assert before stack overflows happen. llvm-svn: 298253
* Strip trailing whitespaceSimon Pilgrim2017-03-201-3/+3
| | | | llvm-svn: 298249
* [IR] Remove some unneeded includes from Operator.h and fix cpp files that ↵Craig Topper2017-03-201-0/+1
| | | | | | were transitively depending on it. NFC llvm-svn: 298235
* Remove unnecessary IDom checkXin Tong2017-03-201-3/+4
| | | | | | | | | | | | | | Summary: This Idom check seems unnecessary. The immediate children of a node on the Dominator Tree should always be the IDom of its immediate children in this case. Reviewers: hfinkel, majnemer, dberlin Reviewed By: dberlin Subscribers: dberlin, davide, llvm-commits Differential Revision: https://reviews.llvm.org/D26954 llvm-svn: 298232
* [InstCombine] Remove duplicate code in SimplifyDemandedUseBits for URem. NFCCraig Topper2017-03-191-2/+0
| | | | llvm-svn: 298231
* Correct a rebase mistake.Xin Tong2017-03-191-2/+2
| | | | | | Left out AA in jumpthreading SimplifyPartiallyRedundantLoad llvm-svn: 298219
* [JumpThreading] Perform phi-translation in SimplifyPartiallyRedundantLoad.Xin Tong2017-03-191-18/+33
| | | | | | | | | | | | | | | | | | | Summary: In case we are loading on a phi-load in SimplifyPartiallyRedundantLoad. Try to phi translate it into incoming values in the predecessors before we search for available loads. This needs https://reviews.llvm.org/D30524 Reviewers: davide, sanjoy, efriedma, dberlin, rengolin Reviewed By: dberlin Subscribers: junbuml, llvm-commits Differential Revision: https://reviews.llvm.org/D30543 llvm-svn: 298217
* [InstCombine] Use setHighBits/setLowBits/setBitsFrom in place of ↵Craig Topper2017-03-191-17/+14
| | | | | | getLowBitsSet/getHighBitsSet. llvm-svn: 298204
* NewGVN: Now that we have a better verifier, we can prove that we can erase ↵Daniel Berlin2017-03-191-1/+3
| | | | | | the predicateuser set each time we mark it touched llvm-svn: 298199
* NewGVN: Remove dead code (for now)Daniel Berlin2017-03-191-4/+0
| | | | llvm-svn: 298198
* [GVN] Fix accidental double storage of the function BasicBlock list in ↵Craig Topper2017-03-181-13/+4
| | | | | | | | | | | | | | | | | | | | | iterateOnFunction Summary: iterateOnFunction creates a ReversePostOrderTraversal object which does a post order traversal in its constructor and stores the results in an internal vector. Iteration over it just reads from the internal vector in reverse order. The GVN code seems to be unaware of this and iterates over ReversePostOrderTraversal object and makes a copy of the vector into a local vector. (I think at one point in time we used a DFS here instead which would have required the local vector). The net affect of this is that we have two vectors containing the basic block list. As I didn't want to expose the implementation detail of ReversePostOrderTraversal's constructor to GVN, I've changed the code to do an explicit post order traversal storing into the local vector and then reverse iterate over that. I've also removed the reserve(256) since the ReversePostOrderTraversal wasn't doing that. I can add it back if we thinks it important. Though it seemed weird that it wasn't based on the size of the function. Reviewers: davide, anemet, dberlin Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D31084 llvm-svn: 298191
* NewGVN: Greatly enhance the ability of the NewGVN verifier to detectDaniel Berlin2017-03-181-86/+150
| | | | | | issues, subsuming previous verifier. llvm-svn: 298188
* NewGVN: Fix PHI evaluation bug exposed by new verifier. We were checking ↵Daniel Berlin2017-03-181-4/+6
| | | | | | whether the incoming block was reachable instead of whether the specific edge was reachable llvm-svn: 298187
* [BuildLibCalls] emitPutChar should infer function attributes for putcharCraig Topper2017-03-171-0/+1
| | | | | | | | | | When InstCombine calls into SimplifyLibCalls and it createa putChar calls, we don't infer the attributes. And since SimplifyLibCalls doesn't use InstCombine's IRBuilder the calls doesn't end up in the worklist on this iteration of InstCombine. So it gets picked up on the next iteration where it causes an IR change. This of course causes InstCombine to run another iteration. So this patch just gets the attributes right the first time. We already did this for puts and some other libcalls. Differential Revision: https://reviews.llvm.org/D31094 llvm-svn: 298171
* [asan] Fix dead stripping of globals on Linux.Evgeniy Stepanov2017-03-173-48/+136
| | | | | | | | | | | | | | | | | | | | | Use a combination of !associated, comdat, @llvm.compiler.used and custom sections to allow dead stripping of globals and their asan metadata. Sometimes. Currently this works on LLD, which supports SHF_LINK_ORDER with sh_link pointing to the associated section. This also works on BFD, which seems to treat comdats as all-or-nothing with respect to linker GC. There is a weird quirk where the "first" global in each link is never GC-ed because of the section symbols. At this moment it does not work on Gold (as in the globals are never stripped). Differential Revision: https://reviews.llvm.org/D30121 llvm-svn: 298158
* [coverity] Fix uninit variable.Vassil Vassilev2017-03-171-1/+1
| | | | | | Patch by John Harvey! llvm-svn: 298122
* [PGO] Change the internal options description. nfc.Rong Xu2017-03-171-3/+5
| | | | llvm-svn: 298120
* [PGO] Value profile for size of memory intrinsic callsRong Xu2017-03-171-5/+22
| | | | | | | | This patch annotates the valuesites profile to memory intrinsics. Differential Revision: http://reviews.llvm.org/D31002 llvm-svn: 298110
* Only unswitch loops with uniform conditionsStanislav Mekhanoshin2017-03-172-6/+22
| | | | | | | | | | | | | | | | | | Loop unswitching can be extremely harmful for a SIMT target. In case if hoisted condition is not uniform a SIMT machine will execute both clones of a loop sequentially. Therefor LoopUnswitch checks if the condition is non-divergent. Since DivergenceAnalysis adds an expensive PostDominatorTree analysis not needed for non-SIMT targets a new option is added to avoid unneded analysis initialization. The method getAnalysisUsage is called when TargetTransformInfo is not yet available and we cannot use it here. For that reason a new field DivergentTarget is added to PassManagerBuilder to control the behavior and set this field from a target. Differential Revision: https://reviews.llvm.org/D30796 llvm-svn: 298104
* [RSForGC] Handle vector GEPsSanjoy Das2017-03-171-0/+5
| | | | | | | | | We were not handling getelemenptr instructions of vector type before. Since getelemenptr instructions for vector types follow the same rule as getelementptr instructions for non-vector types, we can just handle them in the same way. llvm-svn: 298028
* Remove getArgumentList() in favor of arg_begin(), args(), etcReid Kleckner2017-03-165-9/+8
| | | | | | | | | | | | | | | | | Users often call getArgumentList().size(), which is a linear way to get the number of function arguments. arg_size(), on the other hand, is constant time. In general, the fact that arguments are stored in an iplist is an implementation detail, so I've removed it from the Function interface and moved all other users to the argument container APIs (arg_begin(), arg_end(), args(), arg_size()). Reviewed By: chandlerc Differential Revision: https://reviews.llvm.org/D31052 llvm-svn: 298010
* Resubmit r297897: [PGO] Value profile for size of memory intrinsic callsRong Xu2017-03-162-14/+161
| | | | | | | R297897 inadvertently enabled annotation for memop profiling. This new patch fixed it. llvm-svn: 297996
* Salvage debug info from instructions about to be deletedAdrian Prantl2017-03-162-24/+71
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | [Reapplies r297971 and punting on finding a better API for findDbgValues()] This patch improves debug info quality in InstCombine by looking at values that are about to be deleted, checking whether there are any dbg.value instrinsics referring to them, and potentially encoding the semantics of the deleted instruction into the dbg.value's DIExpression. In the example in the testcase (which was extracted from XNU) there is a sequence of %4 = load %struct.entry*, %struct.entry** %next2, align 8, !dbg !41 %5 = bitcast %struct.entry* %4 to i8*, !dbg !42 %add.ptr4 = getelementptr inbounds i8, i8* %5, i64 -8, !dbg !43 %6 = bitcast i8* %add.ptr4 to %struct.entry*, !dbg !44 call void @llvm.dbg.value(metadata %struct.entry* %6, i64 0, metadata !20, metadata !21), !dbg 34 When these instructions are eliminated by instcombine one after another, we can still salvage the otherwise dead debug info: - Bitcasts have no effect, so have the dbg.value point to operand(0) - Loads can be expressed via a DW_OP_deref - Constant gep instructions can be replaced by DWARF expression arithmetic The API introduced by this patch is not specific to instcombine and can be useful in other places, too. rdar://problem/30725338 Differential Revision: https://reviews.llvm.org/D30919 llvm-svn: 297994
* [LoopUnroll] Don't peel loops where the latch isn't the exiting blockMichael Kuperstein2017-03-161-0/+7
| | | | | | | | | Peeling assumed this doesn't happen, but didn't check it. This fixes PR32178. Differential Revision: https://reviews.llvm.org/D30757 llvm-svn: 297993
* [InstCombine] avoid breaking up bitcasted vector min/max patterns (PR32306)Sanjay Patel2017-03-161-0/+10
| | | | | | | | As the related tests show, we're not canonicalizing to this form for scalars or vectors yet, but this solves the immediate problem in: https://bugs.llvm.org/show_bug.cgi?id=32306 llvm-svn: 297989
* Revert commit r297971 because of issues reported by msan.Adrian Prantl2017-03-162-74/+35
| | | | llvm-svn: 297982
* Fix unused variable warnings.Adrian Prantl2017-03-161-3/+3
| | | | llvm-svn: 297973
* Salvage debug info from instructions about to be deletedAdrian Prantl2017-03-162-35/+74
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch improves debug info quality in InstCombine by looking at values that are about to be deleted, checking whether there are any dbg.value instrinsics referring to them, and potentially encoding the semantics of the deleted instruction into the dbg.value's DIExpression. In the example in the testcase (which was extracted from XNU) there is a sequence of %4 = load %struct.entry*, %struct.entry** %next2, align 8, !dbg !41 %5 = bitcast %struct.entry* %4 to i8*, !dbg !42 %add.ptr4 = getelementptr inbounds i8, i8* %5, i64 -8, !dbg !43 %6 = bitcast i8* %add.ptr4 to %struct.entry*, !dbg !44 call void @llvm.dbg.value(metadata %struct.entry* %6, i64 0, metadata !20, metadata !21), !dbg 34 When these instructions are eliminated by instcombine one after another, we can still salvage the otherwise dead debug info: - Bitcasts have no effect, so have the dbg.value point to operand(0) - Loads can be expressed via a DW_OP_deref - Constant gep instructions can be replaced by DWARF expression arithmetic The API introduced by this patch is not specific to instcombine and can be useful in other places, too. rdar://problem/30725338 Differential Revision: https://reviews.llvm.org/D30919 llvm-svn: 297971
* Fix: Refactor SimplifyCFG:canSinkInstructions [NFC]Aditya Kumar2017-03-161-18/+17
| | | | | | Differential Revision: https://reviews.llvm.org/D30116 llvm-svn: 297955
* [InstCombine] Liberate assert in InstCombiner::visitZExtBjorn Pettersson2017-03-161-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The call to canEvaluateZExtd in InstCombiner::visitZExt may return with BitsToClear == SrcTy->getScalarSizeInBits(), but there is an assert that BitsToClear should be smaller than SrcTy->getScalarSizeInBits(). I have a test case that triggers the assert, but it only happens for my downstream target. I've not been able to trigger it for any upstream target. The assert triggered for a piece of code such as this %shr1 = lshr i16 undef, 15 ... %shr2 = lshr i16 %shr1, 1 %conv = zext i16 %shr2 to i32 Normally the lshr instructions are constant folded before we visit the zext (that is why it is so hard to reproduce). The original pattern, before instcombine, is of course a lot more complicated in my test case. The shift count in the second lshr is for example determined by the outcome of a PHI instruction. It seems like other rewrites by instcombine leads up to the pattern above. And then the zext is pulled from the worklist, and visited (hitting the assert), before we detect that the lshr instrucions can be constant folded. Anyway, since the canEvaluateZExtd may return with BitsToClear equal to SrcTy->getScalarSizeInBits(), and since the rewrite that converts the expression type to avoid a zero extend works also for the case where SrcBitsKept ends up being zero, then it should be OK to liberate the assert to assert(BitsToClear <= SrcTy->getScalarSizeInBits() && "Unreasonable BitsToClear"); Reviewers: hfinkel Reviewed By: hfinkel Subscribers: hfinkel, llvm-commits Differential Revision: https://reviews.llvm.org/D30993 llvm-svn: 297952
* Revert "[PGO] Value profile for size of memory intrinsic calls"Eric Liu2017-03-162-158/+13
| | | | | | This commit reverts r297897 and r297909. llvm-svn: 297951
* [PM/Inliner] Fix a bug in r297374 where we would leave stale calls inChandler Carruth2017-03-161-0/+6
| | | | | | | the work queue and crash when trying to visit them after deleting the function containing those calls. llvm-svn: 297940
* [ADCE] Remove redundent code [NFC]Tobias Grosser2017-03-161-19/+0
| | | | | | | | | | | | | | | | | | | | | Summary: In commit r289548 ([ADCE] Add code to remove dead branches) a redundant loop nest was accidentally introduced, which implements exactly the same functionality as has already been available right after. This redundancy has been found when inspecting the ADCE code in the context of our recent discussions on post-dominator modeling. This redundant code was also eliminated by r296535 (which sparked the discussion), but only as part of a larger semantic change of the post-dominance modeling. As this redundency in [ADCE] is really just an oversight completely independent of the post-dominance changes under discussion, we remove this redundancy independently. Reviewers: dberlin, david2050 Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D31023 llvm-svn: 297929
* Revert "Revert "[PGO] Minor cleanup for count instruction in ↵Vitaly Buka2017-03-151-1/+4
| | | | | | | | | | SelectInstVisitor."" Previously reverted wrong revision. This reverts commit r297910. llvm-svn: 297911
* Revert "[PGO] Minor cleanup for count instruction in SelectInstVisitor."Vitaly Buka2017-03-151-4/+1
| | | | | | | | Fails LLVMFuzzer.LLVMFuzzer.value-profile-strncmp.test This reverts commit r297892. llvm-svn: 297910
* Fix build failure from r297897.Rong Xu2017-03-151-3/+3
| | | | llvm-svn: 297909
* [PGO] Value profile for size of memory intrinsic callsRong Xu2017-03-152-13/+158
| | | | | | | | | This patch adds the value profile support to profile the size parameter of memory intrinsic calls: memcpy, memcmp, and memmov. Differential Revision: http://reviews.llvm.org/D28965 llvm-svn: 297897
* [PGO] Minor cleanup for count instruction in SelectInstVisitor.Rong Xu2017-03-151-1/+4
| | | | | | | | | | | | | | | | | Summary: NSIs can be double-counted by different operations in SelectInstVisitor. Sink the the update to VM_counting mode only. Also reset the value for each counting operation. Reviewers: davidxl Reviewed By: davidxl Subscribers: xur, llvm-commits Differential Revision: https://reviews.llvm.org/D30999 llvm-svn: 297892
* [EarlyCSE] reduce indent; NFCISanjay Patel2017-03-151-21/+22
| | | | llvm-svn: 297886
* [PGO] Refactor the code for value profile annotationRong Xu2017-03-151-29/+38
| | | | | | | | | This patch refactors the code for value profile annotation to facilitate of adding other kind of value profiles. Differential Revision: http://reviews.llvm.org/D30989 llvm-svn: 297870
* Revert "Refactor SimplifyCFG:canSinkInstructions [NFC]"Eric Liu2017-03-151-23/+23
| | | | | | This reverts commit r297839, which breaks Transforms/SimplifyCFG/sink-common-code.ll llvm-svn: 297845
* Refactor SimplifyCFG:canSinkInstructions [NFC]Aditya Kumar2017-03-151-23/+23
| | | | llvm-svn: 297839
* MemCpyOptimizer: don't create new addrspace castsFiona Glaser2017-03-141-0/+11
| | | | | | | This isn't safe on all targets, and since we don't have a way to know it's safe, avoid doing it for now. llvm-svn: 297788
* SamplePGO ThinLTO ICP fix for local functions.Dehao Chen2017-03-142-1/+10
| | | | | | | | | | | | | | | | | | | | | | | | Summary: In SamplePGO, if the profile is collected from non-LTO binary, and used to drive ThinLTO, the indirect call promotion may fail because ThinLTO adjusts local function names to avoid conflicts. There are two places of where the mismatch can happen: 1. thin-link prepends SourceFileName to front of FuncName to build the GUID (GlobalValue::getGlobalIdentifier). Unlike instrumentation FDO, SamplePGO does not use the PGOFuncName scheme and therefore the indirect call target profile data contains a hash of the OriginalName. 2. backend compiler promotes some local functions to global and appends .llvm.{$ModuleHash} to the end of the FuncName to derive PromotedFunctionName This patch tries at the best effort to find the GUID from the original local function name (in profile), and use that in ICP promotion, and in SamplePGO matching that happens in the backend after importing/inlining: 1. in thin-link, it builds the map from OriginalName to GUID so that when thin-link reads in indirect call target profile (represented by OriginalName), it knows which GUID to import. 2. in backend compiler, if sample profile reader cannot find a profile match for PromotedFunctionName, it will try to find if there is a match for OriginalFunctionName. 3. in backend compiler, we build symbol table entry for OriginalFunctionName and pointer to the same symbol of PromotedFunctionName, so that ICP can find the correct target to promote. Reviewers: mehdi_amini, tejohnson Reviewed By: tejohnson Subscribers: llvm-commits, Prazek Differential Revision: https://reviews.llvm.org/D30754 llvm-svn: 297757
* [InstCombine] improve readability; NFCISanjay Patel2017-03-141-29/+23
| | | | llvm-svn: 297755
* [LV] Refactor cross-iteration phi's back-patching; NFCGil Rapaport2017-03-141-232/+244
| | | | | | | | | | | | | | This patch refactors the PHisToFix loop as follows: - The loop itself now resides in its own method. - The new method iterates on scalar-loop's header; the PHIsToFix map formerly propagated as an output parameter and filled during phi widening is removed. - The code handling reductions is moved into its own method, similar to the existing fixFirstOrderRecurrence(). Differential Revision: https://reviews.llvm.org/D30755 llvm-svn: 297740
* [LV] Refactor Cost Model's selectVectorizationFactor(); NFCAyal Zaks2017-03-141-73/+132
| | | | | | | | | | | Refactoring Cost Model's selectVectorizationFactor() so that it handles only the selection of the best VF from a pre-computed range of candidate VF's, extracting early-exit criteria and the computation of a MaxVF upper-bound to other methods, all driven by a newly introduced LoopVectorizationPlanner. Differential Revision: https://reviews.llvm.org/D30653 llvm-svn: 297737
* Fix typos in ADCE commentsTobias Grosser2017-03-141-7/+7
| | | | llvm-svn: 297726
* [TargetTransformInfo] getIntrinsicInstrCost() scalarization estimation improvedJonas Paulsson2017-03-143-26/+33
| | | | | | | | | | | | | | | | | | | | | getIntrinsicInstrCost() used to only compute scalarization cost based on types. This patch improves this so that the actual arguments are checked when they are available, in order to handle only unique non-constant operands. Tests updates: Analysis/CostModel/X86/arith-fp.ll Transforms/LoopVectorize/AArch64/interleaved_cost.ll Transforms/LoopVectorize/ARM/interleaved_cost.ll The improvement in getOperandsScalarizationOverhead() to differentiate on constants made it necessary to update the interleaved_cost.ll tests even though they do not relate to intrinsics. Review: Hal Finkel https://reviews.llvm.org/D29540 llvm-svn: 297705
OpenPOWER on IntegriCloud