summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Target
Commit message (Collapse)AuthorAgeFilesLines
* [X86] convertToThreeAddress, make sure second operand of SUB32ri is really ↵Craig Topper2019-10-011-0/+4
| | | | | | | | | | | | | an immediate before calling getImm(). It might be a symbol instead. We can't fold those since we can't negate them. Similar for other SUB with immediates. Fixes PR43529. llvm-svn: 373397
* AMDGPU/SILoadStoreOptimizer: Add helper functions for working with CombineInfoTom Stellard2019-10-011-205/+244
| | | | | | | | | | | | | | | | | | Summary: This is a refactoring that will make future improvements to this pass easier. This change should not change the behavior of the pass. Reviewers: arsenm, pendingchaos, rampitec, nhaehnle, vpykhtin Reviewed By: nhaehnle, vpykhtin Subscribers: kzhuravl, jvesely, wdng, yaxunl, dstuttard, tpr, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D65496 llvm-svn: 373366
* AMDGPU/GlobalISel: Increase max legal size to 1024Matt Arsenault2019-10-013-10/+13
| | | | | | | | There are 1024 bit register classes defined for AGPRs. Additionally OpenCL defines vectors up to 16 x i64, and this helps those tests legalize. llvm-svn: 373350
* [X86] Add a VBROADCAST_LOAD ISD opcode representing a scalar load ↵Craig Topper2019-10-016-182/+335
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | broadcasted to a vector. Summary: This adds the ISD opcode and a DAG combine to create it. There are probably some places where we can directly create it, but I'll leave that for future work. This updates all of the isel patterns to look for this new node. I had to add a few additional isel patterns for aligned extloads which we should probably fix with a DAG combine or something. This does mean that the broadcast load folding for avx512 can no longer match a broadcasted aligned extload. There's still some work to do here for combining a broadcast of a broadcast_load. We also need to improve extractelement or demanded vector elements of a broadcast_load. I'll try to get those done before I submit this patch. Reviewers: RKSimon, spatel Reviewed By: RKSimon Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68198 llvm-svn: 373349
* [AMDGPU] Add VerifyScheduling support.Jay Foad2019-10-011-0/+18
| | | | | | | | | | | | | | | | Summary: This is cut and pasted from the corresponding GenericScheduler functions. Reviewers: arsenm, atrick, tstellar, vpykhtin Subscribers: MatzeB, kzhuravl, jvesely, wdng, nhaehnle, yaxunl, dstuttard, tpr, t-tye, hiraditya, javed.absar, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68264 llvm-svn: 373346
* [X86] Consider isCodeGenOnly in the EVEX2VEX pass to make VMAXPD/PS map to ↵Craig Topper2019-10-011-16/+25
| | | | | | | | | | the non-commutable VEX instruction. Use EVEX2VEX override to fix the scalar instructions. Previously the match was ambiguous and VMAXPS/PD and VMAXCPS/PD were mapped to the same VEX instruction. But we should keep the commutableness when change the opcode. llvm-svn: 373303
* [WebAssembly] Make sure EH pads are preferred in sortingHeejin Ahn2019-10-011-0/+1
| | | | | | | | | | | | | | | | | | | | | Summary: In CFGSort, we try to make EH pads have higher priorities as soon as they are ready to be sorted, to prevent creation of unwind destination mismatches in CFGStackify. We did that by making priority queues' comparison function prefer EH pads, but it was possible for an EH pad to be popped from `Preferred` queue and then not sorted immediately and enter `Ready` queue instead in a certain condition. This patch makes sure that special condition does not consider EH pads as its candidates. Reviewers: dschuff Subscribers: sbc100, jgravelle-google, hiraditya, sunfish, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68229 llvm-svn: 373302
* [WebAssembly] Unstackify regs after fixing unwinding mismatchesHeejin Ahn2019-10-012-0/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Fixing unwind mismatches for exception handling can result in splicing existing BBs and moving some of instructions to new BBs. In this case some of stackified def registers in the original BB can be used in the split BB. For example, we have this BB and suppose %r0 is a stackified register. ``` bb.1: %r0 = call @foo ... use %r0 ... ``` After fixing unwind mismatches in CFGStackify, `bb.1` can be split and some instructions can be moved to a newly created BB: ``` bb.1: %r0 = call @foo bb.split (new): ... use %r0 ... ``` In this case we should make %r0 un-stackified, because its use is now in another BB. When spliting a BB, this CL unstackifies all def registers that have uses in the new split BB. Reviewers: dschuff Subscribers: sbc100, jgravelle-google, hiraditya, sunfish, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68218 llvm-svn: 373301
* AMDGPU/GlobalISel: Select s1 src G_SITOFP/G_UITOFPMatt Arsenault2019-10-014-8/+55
| | | | llvm-svn: 373298
* AMDGPU/GlobalISel: Add support for init.exec intrinsicsMatt Arsenault2019-10-016-20/+42
| | | | | | | TThe existing wave32 behavior seems broken and incomplete, but this reproduces it. llvm-svn: 373296
* AMDGPU/GlobalISel: Allow scc/vcc alternative mappings for s1 constantsMatt Arsenault2019-10-011-1/+15
| | | | llvm-svn: 373295
* AMDGPU/GlobalISel: Avoid creating shift of 0 in arg loweringMatt Arsenault2019-10-011-3/+8
| | | | | | | | This is sort of papering over the fact that we don't run a combiner anywhere, but avoiding creating 2 instructions in the first place is easy. llvm-svn: 373293
* TLI: Remove DAG argument from getRegisterByNameMatt Arsenault2019-10-0120-62/+60
| | | | | | | | | | | Replace with the MachineFunction. X86 is the only user, and only uses it for the function. This removes one obstacle from using this in GlobalISel. The other is the more tolerable EVT argument. The X86 use of the function seems questionable to me. It checks hasFP, before frame lowering. llvm-svn: 373292
* AMDGPU/GlobalISel: Select G_UADDO/G_USUBOMatt Arsenault2019-10-013-1/+47
| | | | llvm-svn: 373288
* GlobalISel: Implement widenScalar for G_SITOFP/G_UITOFP sourcesMatt Arsenault2019-10-011-3/+9
| | | | | | Legalize 16-bit G_SITOFP/G_UITOFP for AMDGPU. llvm-svn: 373287
* AMDGPU/GlobalISel: Legalize G_GLOBAL_VALUEMatt Arsenault2019-10-013-8/+105
| | | | | | | Handle other cases besides LDS. Mostly a straight port of the existing handling, without the intermediate custom nodes. llvm-svn: 373286
* [LegacyPassManager] Deprecate the BasicBlockPass/Manager.Alina Sbirlea2019-09-302-48/+51
| | | | | | | | | | | | | | | | | Summary: The BasicBlockManager is potentially broken and should not be used. Replace all uses of the BasicBlockPass with a FunctionBlockPass+loop on blocks. Reviewers: chandlerc Subscribers: jholewinski, sanjoy.google, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68234 llvm-svn: 373254
* [X86] Mask off upper bits of splat element in LowerBUILD_VECTORvXi1 when ↵Craig Topper2019-09-301-2/+12
| | | | | | | | | | | | | | | | forming a SELECT. The i1 scalar would have been type legalized to i8, but that doesn't guarantee anything about the upper bits. If we're going to use it as condition we need to make sure the upper bits are 0. I've special cased ISD::SETCC conditions since that should guarantee zero upper bits. We could go further and use computeKnownBits, but we have no tests that would need that. Fixes PR43507. llvm-svn: 373246
* [X86] Address post-commit review from code I accidentally commited in r373136.Craig Topper2019-09-301-3/+6
| | | | | | See https://reviews.llvm.org/D68167 llvm-svn: 373245
* [NewPM] Port MachineModuleInfo to the new pass manager.Yuanfang Chen2019-09-302-4/+4
| | | | | | | | | | | | | Existing clients are converted to use MachineModuleInfoWrapperPass. The new interface is for defining a new pass manager API in CodeGen. Reviewers: fedor.sergeev, philip.pfaffe, chandlerc, arsenm Reviewed By: arsenm, fedor.sergeev Differential Revision: https://reviews.llvm.org/D64183 llvm-svn: 373240
* [X86] Add ANY_EXTEND to switch in ReplaceNodeResults, but just fall back to ↵Craig Topper2019-09-301-0/+6
| | | | | | | | | | | | | default handling. ANY_EXTEND of v8i8 is marked Custom on AVX512 for handling extends from v8i8. But the type legalization infrastructure will call ReplaceNodeResults for v8i8 results. We should just defer it the default handling instead of asserting in the default of the switch. Fixes PR43509. llvm-svn: 373234
* [AArch64][SVE] Implement punpk[hi|lo] intrinsicsKerry McLaughlin2019-09-302-2/+15
| | | | | | | | | | | | | | | | | | | | | | Summary: Adds the following two intrinsics: - int_aarch64_sve_punpkhi - int_aarch64_sve_punpklo This patch also contains a fix which allows LLVMHalfElementsVectorType to forward reference overloadable arguments. Reviewers: sdesmalen, rovka, rengolin Reviewed By: sdesmalen Subscribers: tschuett, kristof.beyls, hiraditya, rkruppe, psnobl, greened, cfe-commits, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D67830 llvm-svn: 373232
* [AArch64][GlobalISel] Support lowering variadic musttail callsJessica Paquette2019-09-301-11/+69
| | | | | | | | | | | | | | | | | | | | | | | | | | This adds support for lowering variadic musttail calls. To do this, we have to... - Detect a musttail call in a variadic function before attempting to lower the call's formal arguments. This is done in the IRTranslator. - Compute forwarded registers in `lowerFormalArguments`, and add copies for those registers. - Restore the forwarded registers in `lowerTailCall`. Because there doesn't seem to be any nice way to wrap these up into the outgoing argument handler, the restore code in `lowerTailCall` is done separately. Also, irritatingly, you have to make sure that the registers don't overlap with any passed parameters. Otherwise, the scheduler doesn't know what to do with the extra copies and asserts. Add call-translator-variadic-musttail.ll to test this. This is pretty much the same as the X86 musttail-varargs.ll test. We didn't have as nice of a test to base this off of, but the idea is the same. Differential Revision: https://reviews.llvm.org/D68043 llvm-svn: 373226
* [mips] Fix code indentation. NFCSimon Atanasyan2019-09-301-3/+3
| | | | llvm-svn: 373225
* [AMDGPU] SIFoldOperands should not fold register acrocc the EXEC definitionAlexander Timofeev2019-09-301-0/+7
| | | | | | | | Reviewers: rampitec Differential Revision: https://reviews.llvm.org/D67662 llvm-svn: 373221
* [Alignment][NFC] Remove AllocaInst::setAlignment(unsigned)Guillaume Chatelet2019-09-303-6/+7
| | | | | | | | | | | | | | | | | Summary: This is patch is part of a series to introduce an Alignment type. See this thread for context: http://lists.llvm.org/pipermail/llvm-dev/2019-July/133851.html See this patch for the introduction of the type: https://reviews.llvm.org/D64790 Reviewers: courbet Subscribers: jholewinski, arsenm, jvesely, nhaehnle, eraman, hiraditya, cfe-commits, llvm-commits Tags: #clang, #llvm Differential Revision: https://reviews.llvm.org/D68141 llvm-svn: 373207
* [ARM][MVE] Change VCTP operandSam Parker2019-09-301-3/+3
| | | | | | | | | | | | The VCTP instruction will calculate the predicate masked based upon the number of elements that need to be processed. I had inserted the sub before the vctp intrinsic and supplied it as the operand, but this is incorrect as the phi should directly feed the vctp. The sub is calculating the value for the next iteration. Differential Revision: https://reviews.llvm.org/D67921 llvm-svn: 373188
* [ARM][CGP] Allow signext argumentsSam Parker2019-09-301-5/+2
| | | | | | | | | | | | As we perform a zext on any arguments used in the promoted tree, it doesn't matter if they're marked as signext. The only permitted user(s) in the tree which would interpret the sign bits are signed icmps. For these instructions, their promoted operands are truncated before the icmp uses them. Differential Revision: https://reviews.llvm.org/D68019 llvm-svn: 373186
* [SystemZ] Add SystemZPostRewrite in addPostRegAlloc() instead at -O0.Jonas Paulsson2019-09-301-1/+4
| | | | | | | | SystemZPostRewrite needs to be run before (it may emit COPYs) the Post-RA pseudo pass also at -O0, so it should be added in addPostRegAlloc(). Review: Ulrich Weigand llvm-svn: 373182
* [X86] Remove some redundant isel patterns. NFCICraig Topper2019-09-301-78/+0
| | | | | | | These are all also implemented in avx512_logical_lowering_types with support for masking. llvm-svn: 373181
* AMDGPU/GlobalISel: Fix select for v2s16 and/or/xorMatt Arsenault2019-09-301-15/+17
| | | | llvm-svn: 373180
* [X86] Split v16i32/v8i64 bitreverse on avx512f targets without avx512bw to ↵Craig Topper2019-09-301-1/+12
| | | | | | enable the use of vpshufb on the 256-bit halves. llvm-svn: 373177
* [X86] Fix -Wunused-variable in -DLLVM_ENABLE_ASSERTIONS=off builds after r373174Fangrui Song2019-09-301-0/+1
| | | | llvm-svn: 373175
* [X86] Remove -x86-experimental-vector-widening-legalization command line flagCraig Topper2019-09-292-1257/+145
| | | | | | | | | This was added back to allow some performance regressions to be investigated. The main perf issue was fixed shortly after adding this back and no other major issues have been reported. So I think its safe to remove this again. llvm-svn: 373174
* [X86] Add custom isel logic to match VPTERNLOG from 2 logic ops.Craig Topper2019-09-291-1/+79
| | | | | | | | | | | | | | | There's room from improvement here, but this is a decent starting point. There are a few minor regressions in the vector-rotate tests, where we are now forming a vpternlog from an and before we get a chance to form it for a bitselect that we were matching previously. This results in an AND and an ANDN feeding the vpternlog where previously we just had an AND after the vpternlog. I think we can probably DAG combine the AND with the bitselect to get back to similar codegen. llvm-svn: 373172
* [PowerPC] Fix conditions of assert in PPCAsmPrinterJinsong Ji2019-09-291-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: g++ build emits warning: llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp:667:77: error: suggest parentheses around ?&&? within ?||? [-Werror=parentheses] assert(MO.isGlobal() || MO.isCPI() || MO.isJTI() || MO.isBlockAddress() && ~~~~~~~~~~~~~~~~~~~~^~ "Unexpected operand type for LWZtoc pseudo."); I believe the intension is to assert all different types, so we should add a parentheses to include all '||'. Reviewers: #powerpc, sfertile, hubert.reinterpretcast, Xiangling_L Reviewed By: Xiangling_L Subscribers: wuzish, nemanjai, hiraditya, kbarton, MaskRay, shchenz, steven.zhang, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68180 llvm-svn: 373164
* [ARM] Cortex-M4 schedule additionsDavid Green2019-09-295-17/+40
| | | | | | | | | | | | | | | | | | | This is an attempt to fill in some of the missing instructions from the Cortex-M4 schedule, and make it easier to do the same for other ARM cpus. - Some instructions are marked as hasNoSchedulingInfo as they are pseudos or otherwise do not require scheduling info - A lot of features have been marked not supported - Some WriteRes's have been added for cvt instructions. - Some extra instruction latencies have been added, notably by relaxing the regex for dsp instruction to catch more cases, and some fp instructions. This goes a long way to get the CompleteModel working for this CPU. It does not go far enough as to get all scheduling info for all output operands correct. Differential Revision: https://reviews.llvm.org/D67957 llvm-svn: 373163
* [X86] Enable isel to fold broadcast loads that have been bitcasted from FP ↵Craig Topper2019-09-291-0/+96
| | | | | | into a vpternlog. llvm-svn: 373157
* [X86] Move bitselect matching to vpternlog into X86ISelDAGToDAG.cppCraig Topper2019-09-292-43/+160
| | | | | | | | | | | | This allows us to reduce the use count on the condition node before the match. This enables load folding for that operand without relying on the peephole pass. This will be improved on for broadcast load folding in a subsequent commit. This still requires a bunch of isel patterns for vXi16/vXi8 types though. llvm-svn: 373156
* [X86] Enable canonicalizeBitSelect for AVX512 since we can use VPTERNLOG now.Craig Topper2019-09-291-5/+7
| | | | llvm-svn: 373155
* [X86] Match (or (and A, B), (andn (A, C))) to VPTERNLOG with AVX512.Craig Topper2019-09-291-0/+43
| | | | | | This uses a similar isel pattern as we used for vpcmov with XOP. llvm-svn: 373154
* [GlobalISel Enable memcpy inlining with optsize.Amara Emerson2019-09-281-1/+1
| | | | | | We should be disabling inline for minsize, not optsize. llvm-svn: 373143
* AMDGPU/GlobalISel: Avoid getting MRI in every functionMatt Arsenault2019-09-282-221/+156
| | | | | | | Store it in AMDGPUInstructionSelector to avoid boilerplate in nearly every select function. llvm-svn: 373139
* [X86] Add broadcast load unfolding support for VPTESTMD/Q and VPTESTNMD/Q.Craig Topper2019-09-281-0/+12
| | | | llvm-svn: 373138
* [X86] Stop using UpdateNodeOperands in combineGatherScatter. Create new ↵Craig Topper2019-09-281-35/+58
| | | | | | | | | | | | | | nodes like most other DAG combines. Creating new nodes is what we usually do. Have to explicitly check that we don't update to an existing node and having to manually manage the worklist is unusual. We can probably add a helper function to reduce the duplication of having to check if we should create a gather or scatter, but I wanted to just get the simple thing done. llvm-svn: 373137
* [X86] Split combineGatherScatter into a version for generic ISD nodes and ↵Craig Topper2019-09-281-5/+39
| | | | | | | | | | | | | | | | another version for X86 specific nodes. The majority of the code doesn't run on the X86 nodes today since its gated by isBeforeLegalizeOps and we don't formm X86 nodes until after that. Except for a couple special case in type legalization. But I think we would probably break those if some of the transforms fire on them. I want to remove the hardcoded operand numbers and the unusual use of UpdateNodeOperands. Being able to know which ISD opcodes are present should help with that. llvm-svn: 373136
* [X86] Call SimplifyDemandedBits in combineGatherScatter any time the mask ↵Craig Topper2019-09-271-3/+3
| | | | | | | | | element is wider than i1, not just when AVX512 is disabled. The AVX2 intrinsics can still be used when AVX512 is enabled and those go through this path. So we should simplify them. llvm-svn: 373108
* [RISCV] Rename FPRs and use Register arithmeticLuis Marques2019-09-276-199/+136
| | | | | | | | | | | | | | | | | The new names for FPRs ensure that the Register values within the same class are enumerated consecutively (the order is determined by the `LessRecordRegister` function object). Where there were tables mapping between 32- and 64-bit FPRs (and vice versa) this patch replaces them with Register arithmetic. The enumeration order between different register classes is expected to continue to be arbitrary, although it does impact the conversion from the (overloaded) asm FPR names to Register values, and therefore might require updates to the target if the sorting algorithm is changed. Static asserts were added to ensure that changes to the ordering that would impact the current implementation are detected. Differential Revision: https://reviews.llvm.org/D67423 llvm-svn: 373096
* [AMDGPU][MC] Corrected parsing of registersDmitry Preobrazhensky2019-09-271-144/+208
| | | | | | | | | | | | | Summary of changes: refactored code for better readability and future improvements; fixed bug 41281: https://bugs.llvm.org/show_bug.cgi?id=41281 Reviewers: artem.tamazov, arsenm Differential Revision: https://reviews.llvm.org/D65224 llvm-svn: 373094
* [Alignment][NFC] Remove unneeded llvm:: scoping on Align typesGuillaume Chatelet2019-09-2745-150/+141
| | | | llvm-svn: 373081
OpenPOWER on IntegriCloud