summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
* [coverity] Fix uninit variable.Vassil Vassilev2017-03-171-1/+1
| | | | | | Patch by John Harvey! llvm-svn: 298122
* [PGO] Change the internal options description. nfc.Rong Xu2017-03-171-3/+5
| | | | llvm-svn: 298120
* [PGO] Value profile for size of memory intrinsic callsRong Xu2017-03-171-5/+22
| | | | | | | | This patch annotates the valuesites profile to memory intrinsics. Differential Revision: http://reviews.llvm.org/D31002 llvm-svn: 298110
* Only unswitch loops with uniform conditionsStanislav Mekhanoshin2017-03-172-6/+22
| | | | | | | | | | | | | | | | | | Loop unswitching can be extremely harmful for a SIMT target. In case if hoisted condition is not uniform a SIMT machine will execute both clones of a loop sequentially. Therefor LoopUnswitch checks if the condition is non-divergent. Since DivergenceAnalysis adds an expensive PostDominatorTree analysis not needed for non-SIMT targets a new option is added to avoid unneded analysis initialization. The method getAnalysisUsage is called when TargetTransformInfo is not yet available and we cannot use it here. For that reason a new field DivergentTarget is added to PassManagerBuilder to control the behavior and set this field from a target. Differential Revision: https://reviews.llvm.org/D30796 llvm-svn: 298104
* [RSForGC] Handle vector GEPsSanjoy Das2017-03-171-0/+5
| | | | | | | | | We were not handling getelemenptr instructions of vector type before. Since getelemenptr instructions for vector types follow the same rule as getelementptr instructions for non-vector types, we can just handle them in the same way. llvm-svn: 298028
* Remove getArgumentList() in favor of arg_begin(), args(), etcReid Kleckner2017-03-165-9/+8
| | | | | | | | | | | | | | | | | Users often call getArgumentList().size(), which is a linear way to get the number of function arguments. arg_size(), on the other hand, is constant time. In general, the fact that arguments are stored in an iplist is an implementation detail, so I've removed it from the Function interface and moved all other users to the argument container APIs (arg_begin(), arg_end(), args(), arg_size()). Reviewed By: chandlerc Differential Revision: https://reviews.llvm.org/D31052 llvm-svn: 298010
* Resubmit r297897: [PGO] Value profile for size of memory intrinsic callsRong Xu2017-03-162-14/+161
| | | | | | | R297897 inadvertently enabled annotation for memop profiling. This new patch fixed it. llvm-svn: 297996
* Salvage debug info from instructions about to be deletedAdrian Prantl2017-03-162-24/+71
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | [Reapplies r297971 and punting on finding a better API for findDbgValues()] This patch improves debug info quality in InstCombine by looking at values that are about to be deleted, checking whether there are any dbg.value instrinsics referring to them, and potentially encoding the semantics of the deleted instruction into the dbg.value's DIExpression. In the example in the testcase (which was extracted from XNU) there is a sequence of %4 = load %struct.entry*, %struct.entry** %next2, align 8, !dbg !41 %5 = bitcast %struct.entry* %4 to i8*, !dbg !42 %add.ptr4 = getelementptr inbounds i8, i8* %5, i64 -8, !dbg !43 %6 = bitcast i8* %add.ptr4 to %struct.entry*, !dbg !44 call void @llvm.dbg.value(metadata %struct.entry* %6, i64 0, metadata !20, metadata !21), !dbg 34 When these instructions are eliminated by instcombine one after another, we can still salvage the otherwise dead debug info: - Bitcasts have no effect, so have the dbg.value point to operand(0) - Loads can be expressed via a DW_OP_deref - Constant gep instructions can be replaced by DWARF expression arithmetic The API introduced by this patch is not specific to instcombine and can be useful in other places, too. rdar://problem/30725338 Differential Revision: https://reviews.llvm.org/D30919 llvm-svn: 297994
* [LoopUnroll] Don't peel loops where the latch isn't the exiting blockMichael Kuperstein2017-03-161-0/+7
| | | | | | | | | Peeling assumed this doesn't happen, but didn't check it. This fixes PR32178. Differential Revision: https://reviews.llvm.org/D30757 llvm-svn: 297993
* [InstCombine] avoid breaking up bitcasted vector min/max patterns (PR32306)Sanjay Patel2017-03-161-0/+10
| | | | | | | | As the related tests show, we're not canonicalizing to this form for scalars or vectors yet, but this solves the immediate problem in: https://bugs.llvm.org/show_bug.cgi?id=32306 llvm-svn: 297989
* Revert commit r297971 because of issues reported by msan.Adrian Prantl2017-03-162-74/+35
| | | | llvm-svn: 297982
* Fix unused variable warnings.Adrian Prantl2017-03-161-3/+3
| | | | llvm-svn: 297973
* Salvage debug info from instructions about to be deletedAdrian Prantl2017-03-162-35/+74
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch improves debug info quality in InstCombine by looking at values that are about to be deleted, checking whether there are any dbg.value instrinsics referring to them, and potentially encoding the semantics of the deleted instruction into the dbg.value's DIExpression. In the example in the testcase (which was extracted from XNU) there is a sequence of %4 = load %struct.entry*, %struct.entry** %next2, align 8, !dbg !41 %5 = bitcast %struct.entry* %4 to i8*, !dbg !42 %add.ptr4 = getelementptr inbounds i8, i8* %5, i64 -8, !dbg !43 %6 = bitcast i8* %add.ptr4 to %struct.entry*, !dbg !44 call void @llvm.dbg.value(metadata %struct.entry* %6, i64 0, metadata !20, metadata !21), !dbg 34 When these instructions are eliminated by instcombine one after another, we can still salvage the otherwise dead debug info: - Bitcasts have no effect, so have the dbg.value point to operand(0) - Loads can be expressed via a DW_OP_deref - Constant gep instructions can be replaced by DWARF expression arithmetic The API introduced by this patch is not specific to instcombine and can be useful in other places, too. rdar://problem/30725338 Differential Revision: https://reviews.llvm.org/D30919 llvm-svn: 297971
* Fix: Refactor SimplifyCFG:canSinkInstructions [NFC]Aditya Kumar2017-03-161-18/+17
| | | | | | Differential Revision: https://reviews.llvm.org/D30116 llvm-svn: 297955
* [InstCombine] Liberate assert in InstCombiner::visitZExtBjorn Pettersson2017-03-161-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The call to canEvaluateZExtd in InstCombiner::visitZExt may return with BitsToClear == SrcTy->getScalarSizeInBits(), but there is an assert that BitsToClear should be smaller than SrcTy->getScalarSizeInBits(). I have a test case that triggers the assert, but it only happens for my downstream target. I've not been able to trigger it for any upstream target. The assert triggered for a piece of code such as this %shr1 = lshr i16 undef, 15 ... %shr2 = lshr i16 %shr1, 1 %conv = zext i16 %shr2 to i32 Normally the lshr instructions are constant folded before we visit the zext (that is why it is so hard to reproduce). The original pattern, before instcombine, is of course a lot more complicated in my test case. The shift count in the second lshr is for example determined by the outcome of a PHI instruction. It seems like other rewrites by instcombine leads up to the pattern above. And then the zext is pulled from the worklist, and visited (hitting the assert), before we detect that the lshr instrucions can be constant folded. Anyway, since the canEvaluateZExtd may return with BitsToClear equal to SrcTy->getScalarSizeInBits(), and since the rewrite that converts the expression type to avoid a zero extend works also for the case where SrcBitsKept ends up being zero, then it should be OK to liberate the assert to assert(BitsToClear <= SrcTy->getScalarSizeInBits() && "Unreasonable BitsToClear"); Reviewers: hfinkel Reviewed By: hfinkel Subscribers: hfinkel, llvm-commits Differential Revision: https://reviews.llvm.org/D30993 llvm-svn: 297952
* Revert "[PGO] Value profile for size of memory intrinsic calls"Eric Liu2017-03-162-158/+13
| | | | | | This commit reverts r297897 and r297909. llvm-svn: 297951
* [PM/Inliner] Fix a bug in r297374 where we would leave stale calls inChandler Carruth2017-03-161-0/+6
| | | | | | | the work queue and crash when trying to visit them after deleting the function containing those calls. llvm-svn: 297940
* [ADCE] Remove redundent code [NFC]Tobias Grosser2017-03-161-19/+0
| | | | | | | | | | | | | | | | | | | | | Summary: In commit r289548 ([ADCE] Add code to remove dead branches) a redundant loop nest was accidentally introduced, which implements exactly the same functionality as has already been available right after. This redundancy has been found when inspecting the ADCE code in the context of our recent discussions on post-dominator modeling. This redundant code was also eliminated by r296535 (which sparked the discussion), but only as part of a larger semantic change of the post-dominance modeling. As this redundency in [ADCE] is really just an oversight completely independent of the post-dominance changes under discussion, we remove this redundancy independently. Reviewers: dberlin, david2050 Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D31023 llvm-svn: 297929
* Revert "Revert "[PGO] Minor cleanup for count instruction in ↵Vitaly Buka2017-03-151-1/+4
| | | | | | | | | | SelectInstVisitor."" Previously reverted wrong revision. This reverts commit r297910. llvm-svn: 297911
* Revert "[PGO] Minor cleanup for count instruction in SelectInstVisitor."Vitaly Buka2017-03-151-4/+1
| | | | | | | | Fails LLVMFuzzer.LLVMFuzzer.value-profile-strncmp.test This reverts commit r297892. llvm-svn: 297910
* Fix build failure from r297897.Rong Xu2017-03-151-3/+3
| | | | llvm-svn: 297909
* [PGO] Value profile for size of memory intrinsic callsRong Xu2017-03-152-13/+158
| | | | | | | | | This patch adds the value profile support to profile the size parameter of memory intrinsic calls: memcpy, memcmp, and memmov. Differential Revision: http://reviews.llvm.org/D28965 llvm-svn: 297897
* [PGO] Minor cleanup for count instruction in SelectInstVisitor.Rong Xu2017-03-151-1/+4
| | | | | | | | | | | | | | | | | Summary: NSIs can be double-counted by different operations in SelectInstVisitor. Sink the the update to VM_counting mode only. Also reset the value for each counting operation. Reviewers: davidxl Reviewed By: davidxl Subscribers: xur, llvm-commits Differential Revision: https://reviews.llvm.org/D30999 llvm-svn: 297892
* [EarlyCSE] reduce indent; NFCISanjay Patel2017-03-151-21/+22
| | | | llvm-svn: 297886
* [PGO] Refactor the code for value profile annotationRong Xu2017-03-151-29/+38
| | | | | | | | | This patch refactors the code for value profile annotation to facilitate of adding other kind of value profiles. Differential Revision: http://reviews.llvm.org/D30989 llvm-svn: 297870
* Revert "Refactor SimplifyCFG:canSinkInstructions [NFC]"Eric Liu2017-03-151-23/+23
| | | | | | This reverts commit r297839, which breaks Transforms/SimplifyCFG/sink-common-code.ll llvm-svn: 297845
* Refactor SimplifyCFG:canSinkInstructions [NFC]Aditya Kumar2017-03-151-23/+23
| | | | llvm-svn: 297839
* MemCpyOptimizer: don't create new addrspace castsFiona Glaser2017-03-141-0/+11
| | | | | | | This isn't safe on all targets, and since we don't have a way to know it's safe, avoid doing it for now. llvm-svn: 297788
* SamplePGO ThinLTO ICP fix for local functions.Dehao Chen2017-03-142-1/+10
| | | | | | | | | | | | | | | | | | | | | | | | Summary: In SamplePGO, if the profile is collected from non-LTO binary, and used to drive ThinLTO, the indirect call promotion may fail because ThinLTO adjusts local function names to avoid conflicts. There are two places of where the mismatch can happen: 1. thin-link prepends SourceFileName to front of FuncName to build the GUID (GlobalValue::getGlobalIdentifier). Unlike instrumentation FDO, SamplePGO does not use the PGOFuncName scheme and therefore the indirect call target profile data contains a hash of the OriginalName. 2. backend compiler promotes some local functions to global and appends .llvm.{$ModuleHash} to the end of the FuncName to derive PromotedFunctionName This patch tries at the best effort to find the GUID from the original local function name (in profile), and use that in ICP promotion, and in SamplePGO matching that happens in the backend after importing/inlining: 1. in thin-link, it builds the map from OriginalName to GUID so that when thin-link reads in indirect call target profile (represented by OriginalName), it knows which GUID to import. 2. in backend compiler, if sample profile reader cannot find a profile match for PromotedFunctionName, it will try to find if there is a match for OriginalFunctionName. 3. in backend compiler, we build symbol table entry for OriginalFunctionName and pointer to the same symbol of PromotedFunctionName, so that ICP can find the correct target to promote. Reviewers: mehdi_amini, tejohnson Reviewed By: tejohnson Subscribers: llvm-commits, Prazek Differential Revision: https://reviews.llvm.org/D30754 llvm-svn: 297757
* [InstCombine] improve readability; NFCISanjay Patel2017-03-141-29/+23
| | | | llvm-svn: 297755
* [LV] Refactor cross-iteration phi's back-patching; NFCGil Rapaport2017-03-141-232/+244
| | | | | | | | | | | | | | This patch refactors the PHisToFix loop as follows: - The loop itself now resides in its own method. - The new method iterates on scalar-loop's header; the PHIsToFix map formerly propagated as an output parameter and filled during phi widening is removed. - The code handling reductions is moved into its own method, similar to the existing fixFirstOrderRecurrence(). Differential Revision: https://reviews.llvm.org/D30755 llvm-svn: 297740
* [LV] Refactor Cost Model's selectVectorizationFactor(); NFCAyal Zaks2017-03-141-73/+132
| | | | | | | | | | | Refactoring Cost Model's selectVectorizationFactor() so that it handles only the selection of the best VF from a pre-computed range of candidate VF's, extracting early-exit criteria and the computation of a MaxVF upper-bound to other methods, all driven by a newly introduced LoopVectorizationPlanner. Differential Revision: https://reviews.llvm.org/D30653 llvm-svn: 297737
* Fix typos in ADCE commentsTobias Grosser2017-03-141-7/+7
| | | | llvm-svn: 297726
* [TargetTransformInfo] getIntrinsicInstrCost() scalarization estimation improvedJonas Paulsson2017-03-143-26/+33
| | | | | | | | | | | | | | | | | | | | | getIntrinsicInstrCost() used to only compute scalarization cost based on types. This patch improves this so that the actual arguments are checked when they are available, in order to handle only unique non-constant operands. Tests updates: Analysis/CostModel/X86/arith-fp.ll Transforms/LoopVectorize/AArch64/interleaved_cost.ll Transforms/LoopVectorize/ARM/interleaved_cost.ll The improvement in getOperandsScalarizationOverhead() to differentiate on constants made it necessary to update the interleaved_cost.ll tests even though they do not relate to intrinsics. Review: Hal Finkel https://reviews.llvm.org/D29540 llvm-svn: 297705
* AMDGPU: Fold icmp/fcmp into icmp intrinsicMatt Arsenault2017-03-131-0/+87
| | | | | | | The typical use is a library vote function which compares to 0. Fold the user condition into the intrinsic. llvm-svn: 297650
* API gardening: Rename FindAllocaDbgValue to findDbgValue (NFC)Adrian Prantl2017-03-131-8/+6
| | | | | | | | and use have it use SmallVectorImpl. There is nothing specific about allocas in this function. llvm-svn: 297643
* [LV] Set memcheck metadata also for VF==1Gil Rapaport2017-03-131-5/+1
| | | | | | | | This commit is a follow-up on r297580. It fixes the FIXME added temporarily by that commit to keep the removal of Unroller's specialized version of scalarizeInstruction() an NFC. See https://reviews.llvm.org/D30715 for details. llvm-svn: 297610
* [LV] A unified scalarizeInstruction() for Vectorizer and Unroller; NFCGil Rapaport2017-03-121-66/+8
| | | | | | | | | | | | | | | | Unroller's specialized scalarizeInstruction() is mostly duplicating Vectorizer's variant. OTOH Vectorizer's scalarizeInstruction() already supports the special case of VF==1 except for avoiding mask-bit extraction in that case. This patch removes Unroller's specialized version in favor of a unified method. The only functional difference between the two variants seems to be setting memcheck metadata for loads and stores only in Vectorizer's variant, which is a bug in Unroller. To keep this patch an NFC the unified method doesn't set memcheck metadata for VF==1. Differential Revision: https://reviews.llvm.org/D30715 llvm-svn: 297580
* Test commit.Ayal Zaks2017-03-121-0/+1
| | | | llvm-svn: 297579
* Split NewGVN class into a legacy pass and an impl, instead of a merged class.Daniel Berlin2017-03-122-83/+87
| | | | llvm-svn: 297576
* VNCoercion: Make the function signatures all consistentDaniel Berlin2017-03-112-3/+2
| | | | llvm-svn: 297537
* WholeProgramDevirt: Implement export/import support for VCP.Peter Collingbourne2017-03-101-4/+36
| | | | | | Differential Revision: https://reviews.llvm.org/D30017 llvm-svn: 297503
* WholeProgramDevirt: Implement export/import support for unique ret val opt.Peter Collingbourne2017-03-101-13/+80
| | | | | | Differential Revision: https://reviews.llvm.org/D29917 llvm-svn: 297502
* NewGVN: Rename InitialClass to TOP, which is what most people would expect ↵Daniel Berlin2017-03-101-25/+25
| | | | | | it to be called llvm-svn: 297494
* [SLP] Revert everything that has to do with memory access sorting.Michael Kuperstein2017-03-101-122/+57
| | | | | | | | | | | | This reverts r293386, r294027, r294029 and r296411. Turns out the SLP tree isn't actually a "tree" and we don't handle accessing the same packet of loads in several different orders well, causing miscompiles. Revert until we can fix this properly. llvm-svn: 297493
* WholeProgramDevirt: Fixed compilation error under MSVS2015.George Rimar2017-03-101-9/+18
| | | | | | | | | | | | | | | | | | | | | | It was introduced in: r296945 WholeProgramDevirt: Implement exporting for single-impl devirtualization. --------------------- r296939 WholeProgramDevirt: Add any unsuccessful llvm.type.checked.load devirtualizations to the list of llvm.type.test users. --------------------- Microsoft Visual Studio Community 2015 Version 14.0.23107.0 D14REL Does not compile that code without additional brackets, showing multiple error like below: WholeProgramDevirt.cpp(1216): error C2958: the left bracket '[' found at 'c:\access_softek\llvm\lib\transforms\ipo\wholeprogramdevirt.cpp(1216)' was not matched correctly WholeProgramDevirt.cpp(1216): error C2143: syntax error: missing ']' before '}' WholeProgramDevirt.cpp(1216): error C2143: syntax error: missing ';' before '}' WholeProgramDevirt.cpp(1216): error C2059: syntax error: ']' llvm-svn: 297451
* AMDGPU: Fix insertion point when reducing load intrinsicsMatt Arsenault2017-03-101-0/+3
| | | | | | | The insertion point may be later than the next instruction, so it is necessary to set it when replacing the call. llvm-svn: 297439
* Move memory coercion functions from GVN.cpp to VNCoercion.cpp so they can be ↵Daniel Berlin2017-03-103-447/+460
| | | | | | | | | | | | | | | | | | | shared between GVN and NewGVN. Summary: These are the functions used to determine when values of loads can be extracted from stores, etc, and to perform the necessary insertions to do this. There are no changes to the functions themselves except reformatting, and one case where memdep was informed of a removed load (which was pushed into the caller). Reviewers: davide Subscribers: mgorny, llvm-commits, Prazek Differential Revision: https://reviews.llvm.org/D30478 llvm-svn: 297438
* NewGVN: Rewrite DCE during elimination so we do it as well as old GVN did.Daniel Berlin2017-03-102-53/+109
| | | | llvm-svn: 297428
* NewGVN: Rename a few things for clarityDaniel Berlin2017-03-101-40/+45
| | | | llvm-svn: 297427
OpenPOWER on IntegriCloud