summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Target
Commit message (Collapse)AuthorAgeFilesLines
...
* [WebAssembly] Support for binary atomic RMW instructionsHeejin Ahn2018-07-093-6/+444
| | | | | | | | | | | | | | | | | | Summary: This adds support for binary atomic read-modify-write instructions: add, sub, and, or, xor, and xchg. This does not yet support translations of some of LLVM IR atomicrmw instructions (nand, max, min, umax, and umin) that do not have a direct counterpart in wasm instructions. Reviewers: dschuff Subscribers: sbc100, jgravelle-google, sunfish, llvm-commits Differential Revision: https://reviews.llvm.org/D49088 llvm-svn: 336615
* [Power9] Add __float128 builtins for Rounding OperationsStefan Pintilie2018-07-092-0/+22
| | | | | | | | | | | | | | | Added __float128 support for a number of rounding operations: trunc rint nearbyint round floor ceil Differential Revision: https://reviews.llvm.org/D48415 llvm-svn: 336601
* [WebAssembly] Improve readability of load/stores and tests. NFC.Heejin Ahn2018-07-092-99/+89
| | | | | | | | | | | | | | | | | Summary: - Changed variable/function names to be more consistent - Improved comments in test files - Added more tests - Fixed a few typos - Misc. cosmetic changes Reviewers: dschuff Subscribers: sbc100, jgravelle-google, sunfish, llvm-commits Differential Revision: https://reviews.llvm.org/D49087 llvm-svn: 336598
* [Power9] [LLVM] Add __float128 support for trunc to double round to oddStefan Pintilie2018-07-091-1/+4
| | | | | | | | | Add support for this builtin: double builtin_truncf128_round_to_odd(float128) Differential Revision: https://reviews.llvm.org/D48483 llvm-svn: 336595
* [AMDGPU][Waitcnt] fix "comparison of integers of different signs" build errorMark Searles2018-07-091-1/+1
| | | | | | | | | | | | | | | | | | Build error on Android; reported by and fix provided by (thanks) by Mauro Rossi <issor.oruam@gmail.com> Fixes the following building error: external/llvm/lib/Target/AMDGPU/SIInsertWaitcnts.cpp:1903:61: error: comparison of integers of different signs: 'typename iterator_traits<__wrap_iter<MachineBasicBlock **> >::difference_type' (aka 'int') and 'unsigned int' [-Werror,-Wsign-compare] BlockWaitcntProcessedSet.end(), &MBB) < Count)) { ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~ 1 error generated. Differential Revision: https://reviews.llvm.org/D49089 llvm-svn: 336588
* AMDGPU: Force inlining if LDS global address is usedMatt Arsenault2018-07-093-26/+95
| | | | | | | | | | These won't work for the forseeable future. These aren't allowed from OpenCL, but IPO optimizations can make them appear. Also directly set the attributes on functions, regardless of the linkage rather than cloning functions like before. llvm-svn: 336587
* [X86][TLI] DAGCombine: Unfold variable bit-clearing mask to two shifts.Roman Lebedev2018-07-092-0/+16
| | | | | | | | | | | | | | | | | | | | | Summary: This adds a reverse transform for the instcombine canonicalizations that were added in D47980, D47981. As discussed later, that was worse at least for the code size, and potentially for the performance, too. https://rise4fun.com/Alive/Zmpl Reviewers: craig.topper, RKSimon, spatel Reviewed By: spatel Subscribers: reames, llvm-commits Differential Revision: https://reviews.llvm.org/D48768 llvm-svn: 336585
* [Power9] Add __float128 builtins for Round To OddStefan Pintilie2018-07-091-6/+25
| | | | | | | | | | | | GCC has builtins for these round to odd instructions: __float128 __builtin_sqrtf128_round_to_odd (__float128) __float128 __builtin_{add,sub,mul,div}f128_round_to_odd (__float128, __float128) __float128 __builtin_fmaf128_round_to_odd (__float128, __float128, __float128) Differential Revision: https://reviews.llvm.org/D47550 llvm-svn: 336578
* [X86] In combineFMA, make sure we bitcast the result of isFNEG back the ↵Craig Topper2018-07-091-1/+2
| | | | | | | | expected type before creating the new FMA node. Previously, we were creating malformed SDNodes, but nothing noticed because the type constraints prevented isel from noticing. llvm-svn: 336566
* [X86] Remove some patterns that include a bitcast of a floating point load ↵Craig Topper2018-07-092-10/+0
| | | | | | | | to an integer type. DAG combine should have converted the type of the load. llvm-svn: 336557
* [X86] Remove some patterns that seems to be unreachable.Craig Topper2018-07-092-12/+0
| | | | | | | | These patterns mapped (v2f64 (X86vzmovl (v2f64 (scalar_to_vector FR64:$src)))) to a MOVSD and an zeroing XOR. But the complexity of a pattern for (v2f64 (X86vzmovl (v2f64))) that selects MOVQ is artificially and hides this MOVSD pattern. Weirder still, the SSE version of the pattern was explicitly blocked on SSE41, but yet we had copied it to AVX and AVX512. llvm-svn: 336556
* [X86] Remove some seemingly unnecessary AddedComplexity lines.Craig Topper2018-07-091-8/+4
| | | | | | Looking at the generated tables this didn't seem to make an obvious difference in pattern priority. llvm-svn: 336555
* [AArch64][SVE] Asm: Support for CNT(B|H|W|D) and CNTP instructions.Sander de Smalen2018-07-092-13/+72
| | | | | | | | | | | | | | | | | | This patch adds support for the following instructions: CNTB CNTH - Determine the number of active elements implied by CNTW CNTD the named predicate constant, multiplied by an immediate, e.g. cnth x0, vl8, #16 CNTP - Count active predicate elements, e.g. cntp x0, p0, p1.b counts the number of active elements in p1, predicated by p0, and stores the result in x0. llvm-svn: 336552
* [Power9] Add __float128 support for compare operationsStefan Pintilie2018-07-093-2/+75
| | | | | | | | Added handling for the select f128. Differential Revision: https://reviews.llvm.org/D48294 llvm-svn: 336548
* [AArch64][SVE] Asm: Support for remaining shift instructions.Sander de Smalen2018-07-092-26/+127
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch completes support for shifts, which include: - LSL - Logical Shift Left - LSLR - Logical Shift Left, Reversed form - LSR - Logical Shift Right - LSRR - Logical Shift Right, Reversed form - ASR - Arithmetic Shift Right - ASRR - Arithmetic Shift Right, Reversed form - ASRD - Arithmetic Shift Right for Divide In the following variants: - Predicated shift by immediate - ASR, LSL, LSR, ASRD e.g. asr z0.h, p0/m, z0.h, #1 (active lanes of z0 shifted by #1) - Unpredicated shift by immediate - ASR, LSL*, LSR* e.g. asr z0.h, z1.h, #1 (all lanes of z1 shifted by #1, stored in z0) - Predicated shift by vector - ASR, LSL*, LSR* e.g. asr z0.h, p0/m, z0.h, z1.h (active lanes of z0 shifted by z1, stored in z0) - Predicated shift by vector, reversed form - ASRR, LSLR, LSRR e.g. lslr z0.h, p0/m, z0.h, z1.h (active lanes of z1 shifted by z0, stored in z0) - Predicated shift left/right by wide vector - ASR, LSL, LSR e.g. lsl z0.h, p0/m, z0.h, z1.d (active lanes of z0 shifted by wide elements of vector z1) - Unpredicated shift left/right by wide vector - ASR, LSL, LSR e.g. lsl z0.h, z1.h, z2.d (all lanes of z1 shifted by wide elements of z2, stored in z0) *Variants added in previous patches. llvm-svn: 336547
* [mips] Addition of the [d]rem and [d]remu instructionsStefan Maksimovic2018-07-093-25/+116
| | | | | | | | | | | | | Related to http://reviews.llvm.org/D15772 Depends on http://reviews.llvm.org/D16889 Adds [D]REM[U] instructions. Patch By: Srdjan Obucina Contributions from: Simon Dardis Differential Revision: https://reviews.llvm.org/D17036 llvm-svn: 336545
* [AArch64][SVE] Asm: Support for TBL instruction.Sander de Smalen2018-07-092-0/+35
| | | | | | | | | | | Support for SVE's TBL instruction for programmable table lookup/permute using vector of element indices, e.g. tbl z0.d, { z1.d }, z2.d stores elements from z1, indexed by elements from z2, into z0. llvm-svn: 336544
* [AArch64][SVE] Asm: Support for ADR instruction.Sander de Smalen2018-07-094-16/+88
| | | | | | | | | | | | | | | | | | Supporting various addressing modes: - adr z0.s, [z0.s, z0.s] - adr z0.s, [z0.s, z0.s, lsl #<shift>] - adr z0.d, [z0.d, z0.d] - adr z0.d, [z0.d, z0.d, lsl #<shift>] - adr z0.d, [z0.d, z0.d, uxtw #<shift>] - adr z0.d, [z0.d, z0.d, sxtw #<shift>] Reviewers: rengolin, fhahn, SjoerdMeijer, samparker, javed.absar Reviewed By: SjoerdMeijer Differential Revision: https://reviews.llvm.org/D48870 llvm-svn: 336533
* [AArch64][SVE] Asm: Support for UZP and TRN instructions.Sander de Smalen2018-07-091-0/+8
| | | | | | | | | | | | | | This patch adds support for: UZP1 Concatenate even elements from two vectors UZP2 Concatenate odd elements from two vectors TRN1 Interleave even elements from two vectors TRN2 Interleave odd elements from two vectors With variants for both data and predicate vectors, e.g. uzp1 z0.b, z1.b, z2.b trn2 p0.s, p1.s, p2.s llvm-svn: 336531
* [X86] Improve the message for some asserts. Remove an if that is guaranteed ↵Craig Topper2018-07-091-15/+16
| | | | | | | | | | true by said asserts. This replaces some asserts in lowerV2F64VectorShuffle with the similar asserts from lowerVIF64VectorShuffle which are more readable. The original asserts mentioned a blend, but there's no guarantee that it is a blend. Also remove an if that the asserts prove is always true. Mask[0] is always less than 2 and Mask[1] is always at least 2. Therefore (Mask[0] >= 2) + (Mask[1] >= 2) == 1 must wlays be true. llvm-svn: 336517
* [X86] Remove an AddedComplexity line that seems unnecessary.Craig Topper2018-07-081-4/+2
| | | | | | | | It only existed on SSE and AVX version. AVX512 version didn't have it. I checked the generated table and this didn't seem necessary to creat a match preference. llvm-svn: 336516
* [X86][Nearly NFC] Split SHLD/SHRD into their own WriteShiftDouble classRoman Lebedev2018-07-0811-2/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: {F6603964} While there is still some discrepancies within that new group, it is clearly separate from the other shifts. And Agner's tables agree, these double shifts are clearly different from the normal shifts/rotates. I'm guessing `FeatureSlowSHLD` is related. Indeed, a basic sched pair is *not* the /best/ match. But keeping it in the WriteShift is /clearly/ not ideal either. This can and likely will be fine-tuned later. This is purely mechanical change, it does not change any numbers, as the [lack of the change of] mca tests show. Reviewers: craig.topper, RKSimon, andreadb Reviewed By: craig.topper Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D49015 llvm-svn: 336515
* [X86] Enhance combineFMA to look for FNEG behind an EXTRACT_VECTOR_ELT.Craig Topper2018-07-081-1/+13
| | | | llvm-svn: 336514
* [X86][SSE] Combine v16i8 SHL by constants to multipliesSimon Pilgrim2018-07-081-1/+2
| | | | | | | | Pre-AVX512 (which can perform a quick extend/shift/truncate), extending to 2 v8i16 for the PMULLW and then truncating is more performant than relying on the generic PBLENDVB vXi8 shift path and uses a similar amount of mask constant pool data. Differential Revision: https://reviews.llvm.org/D48963 llvm-svn: 336513
* [X86] Set scheduler classes to unsupported. NFCI.Simon Pilgrim2018-07-081-53/+53
| | | | | | While looking at PR36895 I noticed how much of the atom model was still setting schedules for unsupported SSE4+ instructions. llvm-svn: 336512
* [X86][Basically NFC] Sched: split WriteBitScan into WriteBSF/WriteBSR.Roman Lebedev2018-07-0811-46/+56
| | | | | | | | | | | | | | | | | | Summary: Motivation: {F6597954} This only does the mechanical splitting, does not actually change any numbers, as the tests added in previous revision show. Reviewers: craig.topper, RKSimon, courbet Reviewed By: craig.topper Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D48998 llvm-svn: 336511
* [X86] Add back some intrinsic table entries lost in r336506.Craig Topper2018-07-081-0/+6
| | | | llvm-svn: 336508
* [X86] Add new scalar fma intrinsics with rounding mode that use f32/f64 types.Craig Topper2018-07-083-56/+189
| | | | | | | | | | | | | | This allows us to handle masking in a very similar way to the default rounding version that uses llvm.fma. I had to add new rounding mode CodeGenOnly instructions to support isel when we can't find a movss to grab the upper bits from to use the b_Int instruction. Fast-isel tests have been updated to match new clang codegen. We are currently having trouble folding fneg into the new intrinsic. I'm going to correct that in a follow up patch to keep the size of this one down. A future patch will also remove the old intrinsics. llvm-svn: 336506
* [SelectionDAG] Split float and integer isKnownNeverZero testsSimon Pilgrim2018-07-071-6/+10
| | | | | | | | | | Splits off isKnownNeverZeroFloat to handle +/- 0 float cases. This will make it easier to be more aggressive with the integer isKnownNeverZero tests (similar to ValueTracking), use computeKnownBits etc. Differential Revision: https://reviews.llvm.org/D48969 llvm-svn: 336492
* [CostModel][X86] Add SREM/UREM general and constant costs (PR38056)Simon Pilgrim2018-07-071-3/+31
| | | | | | | | | | We penalize general SDIV/UDIV costs but don't do the same for SREM/UREM. This patch makes general vector SREM/UREM x20 as costly as scalar, the same approach as we do for SDIV/UDIV. The patch also extends the existing SDIV/UDIV constant costs for SREM/UREM - at the moment this means the additional cost of a MUL+SUB (see D48975). Differential Revision: https://reviews.llvm.org/D48980 llvm-svn: 336486
* [MachineOutliner] Assert that Liveness tracking is accurate (NFC)Yvan Roux2018-07-071-0/+2
| | | | | | | | | | The checking is done deeper inside MachineBasicBlock, but this will hopefully help to find issues when porting the machine outliner to a target where Liveness tracking is broken (like ARM). Differential Revision: https://reviews.llvm.org/D49023 llvm-svn: 336481
* [X86] Merge INTR_TYPE_3OP_RM with INTR_TYPE_3OP. Remove unused INTR_TYPE_1OP_RM.Craig Topper2018-07-072-40/+21
| | | | llvm-svn: 336476
* Use Type::isIntOrPtrTy where possible, NFCVedant Kumar2018-07-062-2/+2
| | | | | | | | | | | It's a bit neater to write T.isIntOrPtrTy() over `T.isIntegerTy() || T.isPointerTy()`. I used Python's re.sub with this regex to update users: r'([\w.\->()]+)isIntegerTy\(\)\s*\|\|\s*\1isPointerTy\(\)' llvm-svn: 336462
* [X86] Remove patterns for MOVLPD/MOVLPS nodes with integer types.Craig Topper2018-07-061-8/+0
| | | | | | Lowering shouldn't generate these. If we need to use them for integer types, it should use a bitcast. llvm-svn: 336458
* [X86] Add more FMA3 memory folding patterns. Remove patterns that are no ↵Craig Topper2018-07-062-53/+49
| | | | | | | | longer needed. We've removed the legacy FMA3 intrinsics and are now using llvm.fma and extractelement/insertelement. So we don't need patterns for the nodes that could only be created by the old intrinscis. Those ISD opcodes still exist because we haven't dropped the AVX512 intrinsics yet, but those should go to EVEX instructions. llvm-svn: 336457
* AMDGPU: Fix UBSan error caused by r335942Tom Stellard2018-07-063-24/+21
| | | | | | | | | | | | | | Summary: Fixes PR38071. Reviewers: arsenm, dstenb Reviewed By: arsenm Subscribers: kzhuravl, wdng, nhaehnle, yaxunl, dstuttard, tpr, t-tye, llvm-commits Differential Revision: https://reviews.llvm.org/D48979 llvm-svn: 336448
* [ARM] ParallelDSP: added statistics, NFC.Sjoerd Meijer2018-07-061-3/+7
| | | | | | | | | Added statistics for the number of SMLAD instructions created, and als renamed the pass name to -arm-parallel-dsp. Differential Revision: https://reviews.llvm.org/D48971 llvm-svn: 336441
* [AArch64] Armv8.4-A: TLB supportSjoerd Meijer2018-07-062-0/+56
| | | | | | | | This adds: - outer shareable TLB Maintenance instructions, and - TLB range maintenance instructions. llvm-svn: 336434
* Recommit: [AArch64] Armv8.4-A: Flag manipulation instructionsSjoerd Meijer2018-07-063-0/+61
| | | | | | Now with the asm operand definition included. llvm-svn: 336432
* Revert [AArch64] Armv8.4-A: Flag manipulation instructionsSjoerd Meijer2018-07-062-35/+0
| | | | | | It's causing build errors. llvm-svn: 336422
* [AArch64] Armv8.4-A: Flag manipulation instructionsSjoerd Meijer2018-07-062-0/+35
| | | | | | | | These instructions are added to AArch64 only. Differential Revision: https://reviews.llvm.org/D48926 llvm-svn: 336421
* [AArch64][ARM] Armv8.4-A: Trace synchronization barrier instructionSjoerd Meijer2018-07-0612-4/+164
| | | | | | | | This adds the Armv8.4-A Trace synchronization barrier (TSB) instruction. Differential Revision: https://reviews.llvm.org/D48918 llvm-svn: 336418
* [X86] Remove FMA4 scalar intrinsics. Use llvm.fma intrinsic instead.Craig Topper2018-07-065-55/+61
| | | | | | | | The intrinsics can be implemented with a f32/f64 llvm.fma intrinsic and an insert into a zero vector. There are a couple regressions here due to SelectionDAG not being able to pull an fneg through an extract_vector_elt. I'm not super worried about this though as InstCombine should be able to do it before we get to SelectionDAG. llvm-svn: 336416
* [X86] Remove all of the avx512 masked packed fma intrinsics. Use llvm.fma or ↵Craig Topper2018-07-062-96/+1
| | | | | | | | | | unmasked 512-bit intrinsics with rounding mode. This upgrades all of the intrinsics to use fneg instructions to convert fma into fmsub/fnmsub/fnmadd/fmsubadd. And uses a select instruction for masking. This matches how clang uses the intrinsics these days. llvm-svn: 336409
* [Power9] Add __float128 library call for fremStefan Pintilie2018-07-061-0/+2
| | | | | | | | Power 9 does not have a hardware instruction for frem but we can call fmodf128. Differential Revision: https://reviews.llvm.org/D48552 llvm-svn: 336406
* [X86][Disassembler] Fix LOCK prefix disassembler supportMaksim Panchenko2018-07-053-0/+7
| | | | | | | | | | | | | | | | | | | Summary: If LOCK prefix is not the first prefix in an instruction, LLVM disassembler silently drops the prefix. The fix is to select a proper instruction with a builtin LOCK prefix if one exists. Reviewers: craig.topper Reviewed By: craig.topper Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D49001 llvm-svn: 336400
* [WebAssembly] Add missing _S opcodes of atomic stores to InstPrinterHeejin Ahn2018-07-051-0/+7
| | | | | | | | | | | | Summary: This was missing in D48839 (rL336145). Reviewers: aardappel Subscribers: dschuff, sbc100, jgravelle-google, sunfish, llvm-commits Differential Revision: https://reviews.llvm.org/D48992 llvm-svn: 336390
* This is a recommit of r336322, previously reverted in r336324 due toSander de Smalen2018-07-052-1/+16
| | | | | | | | | | | | | | | | | | | | | | a deficiency in TableGen that has been addressed in r336334. [AArch64][SVE] Asm: Support for predicated FP rounding instructions. This patch also adds instructions for predicated FP square-root and reciprocal exponent. The added instructions are: - FRINTI Round to integral value (current FPCR rounding mode) - FRINTX Round to integral value (current FPCR rounding mode, signalling inexact) - FRINTA Round to integral value (to nearest, with ties away from zero) - FRINTN Round to integral value (to nearest, with ties to even) - FRINTZ Round to integral value (toward zero) - FRINTM Round to integral value (toward minus Infinity) - FRINTP Round to integral value (toward plus Infinity) - FSQRT Floating-point square root - FRECPX Floating-point reciprocal exponent llvm-svn: 336387
* [X86] Remove the last of the 'x86.fma.' intrinsics and autoupgrade them to ↵Craig Topper2018-07-051-4/+0
| | | | | | | | 'llvm.fma'. Add upgrade tests for all. Still need to remove the AVX512 masked versions. llvm-svn: 336383
* [X86] Add SHUF128 to target shuffle decoding.Craig Topper2018-07-051-0/+11
| | | | | | Differential Revision: https://reviews.llvm.org/D48954 llvm-svn: 336376
OpenPOWER on IntegriCloud