summaryrefslogtreecommitdiffstats
path: root/llvm/test/CodeGen/AArch64
Commit message (Collapse)AuthorAgeFilesLines
...
* [Aarch64][SVE] Add intrinsics for gather loads (vector + imm)Sander de Smalen2019-12-031-0/+139
| | | | | | | | | | | | | | | | | | This patch adds intrinsics for SVE gather loads from memory addresses generated by a vector base plus immediate index: * @llvm.aarch64.sve.ld1.gather.imm This intrinsics maps 1-1 to the corresponding SVE instruction (example for half-words): * ld1h { z0.d }, p0/z, [z0.d, #16] Committed on behalf of Andrzej Warzynski (andwar) Reviewers: sdesmalen, huntergr, kmclaughlin, eli.friedman, rengolin, rovka, dancgr, mgudim, efriedma Reviewed By: sdesmalen Tags: #llvm Differential Revision: https://reviews.llvm.org/D70806
* Precommit tests for D70673Sanne Wouda2019-12-032-12/+120
|
* [Aarch64][SVE] Add intrinsics for gather loads with 32-bits offsetsSander de Smalen2019-12-032-0/+457
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds intrinsics for SVE gather loads for which the offsets are 32-bits wide and are: * unscaled * @llvm.aarch64.sve.ld1.gather.sxtw * @llvm.aarch64.sve.ld1.gather.uxtw * scaled (offsets become indices) * @llvm.arch64.sve.ld1.gather.sxtw.index * @llvm.arch64.sve.ld1.gather.uxtw.index The offsets are either zero (uxtw) or sign (sxtw) extended to 64 bits. These intrinsics map 1-1 to the corresponding SVE instructions (examples for half-words): * unscaled * ld1h { z0.s }, p0/z, [x0, z0.s, sxtw] * ld1h { z0.s }, p0/z, [x0, z0.s, uxtw] * scaled * ld1h { z0.s }, p0/z, [x0, z0.s, sxtw #1] * ld1h { z0.s }, p0/z, [x0, z0.s, uxtw #1] Committed on behalf of Andrzej Warzynski (andwar) Reviewers: sdesmalen, kmclaughlin, eli.friedman, rengolin, rovka, huntergr, dancgr, mgudim, efriedma Reviewed By: sdesmalen Tags: #llvm Differential Revision: https://reviews.llvm.org/D70782
* [AArch64][SVE2] Implement remaining SVE2 floating-point intrinsicsKerry McLaughlin2019-12-033-0/+357
| | | | | | | | | | | | | | | | | | | Summary: Adds the following intrinsics: - faddp - fmaxp, fminp, fmaxnmp & fminnmp - fmlalb, fmlalt, fmlslb & fmlslt - flogb Reviewers: huntergr, sdesmalen, dancgr, efriedma Reviewed By: sdesmalen Subscribers: efriedma, tschuett, kristof.beyls, hiraditya, cameron.mcinally, cfe-commits, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D70253
* [AArch64][SVE] Add intrinsics for gather loads with 64-bit offsetsSander de Smalen2019-12-032-0/+133
| | | | | | | | | | | | | | | | | | | | This patch adds the following intrinsics for gather loads with 64-bit offsets: * @llvm.aarch64.sve.ld1.gather (unscaled offset) * @llvm.aarch64.sve.ld1.gather.index (scaled offset) These intrinsics map 1-1 to the following AArch64 instructions respectively (examples for half-words): * ld1h { z0.d }, p0/z, [x0, z0.d] * ld1h { z0.d }, p0/z, [x0, z0.d, lsl #1] Committing on behalf of Andrzej Warzynski (andwar) Reviewers: sdesmalen, huntergr, rovka, mgudim, dancgr, rengolin, efriedma Reviewed By: efriedma Tags: #llvm Differential Revision: https://reviews.llvm.org/D70542
* [AArch64][SVE] Implement shift intrinsicsKerry McLaughlin2019-12-031-0/+367
| | | | | | | | | | | | | | | | | | | | Summary: Adds the following intrinsics: - asr & asrd - insr - lsl & lsr This patch also adds a new AArch64ISD node (INSR) to represent the int_aarch64_sve_insr intrinsic. Reviewers: huntergr, sdesmalen, dancgr, mgudim, rengolin, efriedma Reviewed By: sdesmalen Subscribers: tschuett, kristof.beyls, hiraditya, rkruppe, psnobl, cameron.mcinally, cfe-commits, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D70437
* Automatically generated arm64-abi-varargs.ll . NFCAmaury Séchet2019-12-021-28/+122
|
* [GlobalISel] CombinerHelper: Fix a bug in matchCombineCopyVolkan Keles2019-12-021-0/+86
| | | | | | | | | | | | | | | | | Summary: When combining COPY instructions, we were replacing the destination registers with the source register without checking register constraints. This patch adds a simple logic to check if the constraints match before replacing registers. Reviewers: qcolombet, aditya_nandakumar, aemerson, paquette, dsanders, Petar.Avramovic Reviewed By: aditya_nandakumar Subscribers: rovka, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D70616
* [ARM][AArch64] Complex addition Neon intrinsics for Armv8.3-AVictor Campos2019-12-021-0/+67
| | | | | | | | | | | | | | | | | | | Summary: Add support for vcadd_* family of intrinsics. This set of intrinsics is available in Armv8.3-A. The fp16 versions require the FP16 extension, which has been available (opt-in) since Armv8.2-A. Reviewers: t.p.northover Reviewed By: t.p.northover Subscribers: t.p.northover, kristof.beyls, hiraditya, cfe-commits, llvm-commits Tags: #clang, #llvm Differential Revision: https://reviews.llvm.org/D70862
* Revert 651f07908a1 "[AArch64] Don't combine callee-save and local stack ↵Hans Wennborg2019-11-301-25/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | adjustment when optimizing for size" This caused asserts (and perhaps also miscompiles) while building for Windows on AArch64. See the discussion on D68530 for details and reproducer. Reverting until this can be investigated and fixed. > For arm64, D18619 introduced the ability to combine bumping the stack pointer > upfront in case it needs to be bumped for both the callee-save area as well as > the local stack area. > > That diff already remarks that "This change can cause an increase in > instructions", but argues that even when that happens, it should be still be a > performance benefit because the number of micro-ops is reduced. > > We have observed that this code-size increase can be significant in practice. > This diff disables combining stack bumping for methods that are marked as > optimize-for-size. > > Example of a prologue with the behavior before this diff (combining stack bumping when possible): > sub sp, sp, #0x40 > stp d9, d8, [sp, #0x10] > stp x20, x19, [sp, #0x20] > stp x29, x30, [sp, #0x30] > add x29, sp, #0x30 > [... compute x8 somehow ...] > stp x0, x8, [sp] > > And after this diff, if the method is marked as optimize-for-size: > stp d9, d8, [sp, #-0x30]! > stp x20, x19, [sp, #0x10] > stp x29, x30, [sp, #0x20] > add x29, sp, #0x20 > [... compute x8 somehow ...] > stp x0, x8, [sp, #-0x10]! > > Note that without combining the stack bump there are two auto-decrements, > nicely folded into the stp instructions, whereas otherwise there is a single > sub sp, ... instruction, but not folded. > > Patch by Nikolai Tillmann! > > Differential Revision: https://reviews.llvm.org/D68530
* [AArch64][SVE] Implement floating-point conversion intrinsicsKerry McLaughlin2019-11-262-0/+484
| | | | | | | | | | | | | | | | | | | | Summary: Adds intrinsics for the following: - fcvt - fcvtzs & fcvtzu - scvtf & ucvtf - fcvtlt, fcvtnt - fcvtx & fcvtxnt Reviewers: huntergr, sdesmalen, dancgr, mgudim, efriedma Reviewed By: sdesmalen Subscribers: tschuett, kristof.beyls, hiraditya, rkruppe, psnobl, cameron.mcinally, cfe-commits, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D70180
* Reland "[DAGCombiner] Allow zextended load combines."Clement Courbet2019-11-222-16/+8
| | | | Check that the generated type is simple.
* [Codegen] TargetLowering::prepareUREMEqFold(): `x u% C1 ==/!= C2` (PR35479)Roman Lebedev2019-11-222-103/+96
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The current lowering is: ``` Name: (X % C1) == C2 -> X * C3 <= C4 || false Pre: (C2 == 0 || C1 u<= C2) && (C1 u>> countTrailingZeros(C1)) * C3 == 1 %zz = and i8 C3, 0 ; trick alive into making C3 avaliable in precondition %o0 = urem i8 %x, C1 %r = icmp eq i8 %o0, C2 => %zz = and i8 C3, 0 ; and silence it from complaining about said reg %C4 = -1 /u C1 %n0 = mul i8 %x, C3 %n1 = lshr i8 %n0, countTrailingZeros(C1) ; rotate right %n2 = shl i8 %n0, ((8-countTrailingZeros(C1)) %u 8) ; rotate right %n3 = or i8 %n1, %n2 ; rotate right %is_tautologically_false = icmp ule i8 C1, C2 %C4_fixed = select i1 %is_tautologically_false, i8 -1, i8 %C4 %res = icmp ule i8 %n3, %C4_fixed %r = xor i1 %res, %is_tautologically_false ``` https://rise4fun.com/Alive/2xC https://rise4fun.com/Alive/jpb5 However, we can support non-tautological cases `C1 u> C2` too. Said handling consists of two parts: * `C2 u<= (-1 %u C1)`. It just works. We only have to change `(X % C1) == C2` into `((X - C2) % C1) == 0` ``` Name: (X % C1) == C2 -> (X - C2) * C3 <= C4 iff C2 u<= (-1 %u C1) Pre: (C1 u>> countTrailingZeros(C1)) * C3 == 1 && C2 u<= (-1 %u C1) %zz = and i8 C3, 0 ; trick alive into making C3 avaliable in precondition %o0 = urem i8 %x, C1 %r = icmp eq i8 %o0, C2 => %zz = and i8 C3, 0 ; and silence it from complaining about said reg %C4 = (-1 /u C1) %n0 = sub i8 %x, C2 %n1 = mul i8 %n0, C3 %n2 = lshr i8 %n1, countTrailingZeros(C1) ; rotate right %n3 = shl i8 %n1, ((8-countTrailingZeros(C1)) %u 8) ; rotate right %n4 = or i8 %n2, %n3 ; rotate right %is_tautologically_false = icmp ule i8 C1, C2 %C4_fixed = select i1 %is_tautologically_false, i8 -1, i8 %C4 %res = icmp ule i8 %n4, %C4_fixed %r = xor i1 %res, %is_tautologically_false ``` https://rise4fun.com/Alive/m4P https://rise4fun.com/Alive/SKrx * `C2 u> (-1 %u C1)`. We also have to change `(X % C1) == C2` into `((X - C2) % C1) == 0`, and we have to decrement C4: ``` Name: (X % C1) == C2 -> (X - C2) * C3 <= C4 iff C2 u> (-1 %u C1) Pre: (C1 u>> countTrailingZeros(C1)) * C3 == 1 && C2 u> (-1 %u C1) %zz = and i8 C3, 0 ; trick alive into making C3 avaliable in precondition %o0 = urem i8 %x, C1 %r = icmp eq i8 %o0, C2 => %zz = and i8 C3, 0 ; and silence it from complaining about said reg %C4 = (-1 /u C1)-1 %n0 = sub i8 %x, C2 %n1 = mul i8 %n0, C3 %n2 = lshr i8 %n1, countTrailingZeros(C1) ; rotate right %n3 = shl i8 %n1, ((8-countTrailingZeros(C1)) %u 8) ; rotate right %n4 = or i8 %n2, %n3 ; rotate right %is_tautologically_false = icmp ule i8 C1, C2 %C4_fixed = select i1 %is_tautologically_false, i8 -1, i8 %C4 %res = icmp ule i8 %n4, %C4_fixed %r = xor i1 %res, %is_tautologically_false ``` https://rise4fun.com/Alive/d40 https://rise4fun.com/Alive/8cF I believe this concludes `x u% C1 ==/!= C2` lowering. In fact, clang is may now be better in this regard than gcc: as it can be seen from `@t32_6_4` test, we do lower `x % 6 == 4` via this pattern, while gcc does not: https://godbolt.org/z/XNU2z9 And all the general alive proofs say this is legal. And manual checking agrees: https://rise4fun.com/Alive/WA2 Fixes [[ https://bugs.llvm.org/show_bug.cgi?id=35479 | PR35479 ]]. Reviewers: RKSimon, craig.topper, spatel Reviewed By: RKSimon Subscribers: nick, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D70053
* [Codegen] TargetLowering::prepareUREMEqFold(): `x u% C1 ==/!= C2` with ↵Roman Lebedev2019-11-221-61/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | tautological C1 u<= C2 (PR35479) Summary: This is a preparatory cleanup before i add more of this fold to deal with comparisons with non-zero. In essence, the current lowering is: ``` Name: (X % C1) == 0 -> X * C3 <= C4 Pre: (C1 u>> countTrailingZeros(C1)) * C3 == 1 %zz = and i8 C3, 0 ; trick alive into making C3 avaliable in precondition %o0 = urem i8 %x, C1 %r = icmp eq i8 %o0, 0 => %zz = and i8 C3, 0 ; and silence it from complaining about said reg %C4 = -1 /u C1 %n0 = mul i8 %x, C3 %n1 = lshr i8 %n0, countTrailingZeros(C1) ; rotate right %n2 = shl i8 %n0, ((8-countTrailingZeros(C1)) %u 8) ; rotate right %n3 = or i8 %n1, %n2 ; rotate right %r = icmp ule i8 %n3, %C4 ``` https://rise4fun.com/Alive/oqd It kinda just works, really no weird edge-cases. But it isn't all that great for when comparing with non-zero. In particular, given `(X % C1) == C2`, there will be problems in the always-false tautological case where `C2 u>= C1`: https://rise4fun.com/Alive/pH3 That case is tautological, always-false: ``` Name: (X % Y) u>= Y %o0 = urem i8 %x, %y %r = icmp uge i8 %o0, %y => %r = false ``` https://rise4fun.com/Alive/ofu While we can't/shouldn't get such tautological case normally, we do deal with non-splat vectors, so unless we want to give up in this case, we need to fixup/short-circuit such lanes. There are two lowering variants: 1. We can blend between whatever computed result and the correct tautological result ``` Name: (X % C1) == C2 -> X * C3 <= C4 || false Pre: (C2 == 0 || C1 u<= C2) && (C1 u>> countTrailingZeros(C1)) * C3 == 1 %zz = and i8 C3, 0 ; trick alive into making C3 avaliable in precondition %o0 = urem i8 %x, C1 %r = icmp eq i8 %o0, C2 => %zz = and i8 C3, 0 ; and silence it from complaining about said reg %C4 = -1 /u C1 %n0 = mul i8 %x, C3 %n1 = lshr i8 %n0, countTrailingZeros(C1) ; rotate right %n2 = shl i8 %n0, ((8-countTrailingZeros(C1)) %u 8) ; rotate right %n3 = or i8 %n1, %n2 ; rotate right %is_tautologically_false = icmp ule i8 C1, C2 %res = icmp ule i8 %n3, %C4 %r = select i1 %is_tautologically_false, i1 0, i1 %res ``` https://rise4fun.com/Alive/PjT5 https://rise4fun.com/Alive/1KV 2. We can invert the comparison result ``` Name: (X % C1) == C2 -> X * C3 <= C4 || false Pre: (C2 == 0 || C1 u<= C2) && (C1 u>> countTrailingZeros(C1)) * C3 == 1 %zz = and i8 C3, 0 ; trick alive into making C3 avaliable in precondition %o0 = urem i8 %x, C1 %r = icmp eq i8 %o0, C2 => %zz = and i8 C3, 0 ; and silence it from complaining about said reg %C4 = -1 /u C1 %n0 = mul i8 %x, C3 %n1 = lshr i8 %n0, countTrailingZeros(C1) ; rotate right %n2 = shl i8 %n0, ((8-countTrailingZeros(C1)) %u 8) ; rotate right %n3 = or i8 %n1, %n2 ; rotate right %is_tautologically_false = icmp ule i8 C1, C2 %C4_fixed = select i1 %is_tautologically_false, i8 -1, i8 %C4 %res = icmp ule i8 %n3, %C4_fixed %r = xor i1 %res, %is_tautologically_false ``` https://rise4fun.com/Alive/2xC https://rise4fun.com/Alive/jpb5 3. We can expand into `and`/`or`: https://rise4fun.com/Alive/WGn https://rise4fun.com/Alive/lcb5 Blend-one is likely better since we avoid having to load the replacement from constant pool. `xor` is second best since it's still pretty general. I'm not adding `and`/`or` variants. Reviewers: RKSimon, craig.topper, spatel Reviewed By: RKSimon Subscribers: nick, hiraditya, xbolva00, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D70051
* Revert "[DAGCombiner] Allow zextended load combines."Clement Courbet2019-11-222-8/+16
| | | | Breaks some bots.
* [DAGCombiner] Allow zextended load combines.Clement Courbet2019-11-222-16/+8
| | | | | | | | | | | | Summary: or(zext(load8(base)), zext(load8(base+1)) -> zext(load16 base) Reviewers: apilipenko, RKSimon Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D70487
* [AArch64] [FrameLowering] Allow conditional insertion of CFI instructionDavid Tellenbach2019-11-222-4/+17
| | | | | | | | | | | | | | | | | | Summary: The insertion of most CFI instructions during AArch64 frame lowering can be disabled (e.g. using the function attribute `nounwind`). This patch enables conditional insertion for one more CFI instruction. Reviewers: t.p.northover, ostannard Reviewed By: ostannard Subscribers: kristof.beyls, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D70129
* [CodeGen][NFC] Regenerate load-combine test with update_llc_test.Clement Courbet2019-11-202-133/+175
| | | | To prepare for D27861.
* [AArch64] Fix MIR test instruction to not have invalid operand.Amara Emerson2019-11-191-2/+2
| | | | In anticipation of an improved verifier in D63973.
* MTE: add more unchecked instructions.Evgenii Stepanov2019-11-191-4/+123
| | | | | | | | | | | | | | Summary: In particular, 1- and 2-byte loads and stores ignore the pointer tag when using SP as the base register. Reviewers: pcc, ostannard Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D70341
* arm64_32: support function return in FastISel.Tim Northover2019-11-182-8/+44
|
* [GISel][CombinerHelper] Add support for scalar type for the result of ↵Quentin Colombet2019-11-151-0/+42
| | | | | | | | | | | | | | | | | | shuffle vector LLVM IR of 1-element vectors get lower into scalar in GISel. As a result, shuffle vector may also produce a scalar. This patch teaches the shuffle combiner how to deal with scalars when they are in the destination type of a shuffle vector. For now, we just support the easy case where this can be lowered to a plain copy. For other cases, we leave the shuffle vector as is. This type of IR are seen in O0 pipelines. E.g., as produced with SingleSource/UnitTests/Vector/AArch64/aarch64_neon_intrinsics.c. rdar://problem/57198904
* [globalisel][irtanslator] The IRTranslator should preserve TBAA informationDaniel Sanders2019-11-141-0/+19
|
* [AArch64][SVE] Implement floating-point comparison & reduction intrinsicsKerry McLaughlin2019-11-142-0/+481
| | | | | | | | | | | | | | | | | | | Summary: Adds intrinsics for the following: - fadda & faddv - fminv, fmaxv, fminnmv & fmaxnmv - facge & facgt - fcmp[eq|ge|gt|ne|uo] Reviewers: sdesmalen, huntergr, dancgr, mgudim Reviewed By: sdesmalen Subscribers: tschuett, kristof.beyls, hiraditya, rkruppe, psnobl, cameron.mcinally, cfe-commits, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D69858
* [AArch64][SVE] Implement remaining floating-point arithmetic intrinsicsKerry McLaughlin2019-11-141-19/+533
| | | | | | | | | | | | | | | | | | | | Summary: Adds intrinsics for the following: - fabs & fneg - fexpa - frint[a|i|m|n|p|x|z] - frecpe, frecps & frecpx - fsqrt, frsqrte & frsqrts Reviewers: huntergr, sdesmalen, dancgr, mgudim Reviewed By: sdesmalen Subscribers: tschuett, kristof.beyls, hiraditya, rkruppe, psnobl, cameron.mcinally, cfe-commits, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D69800
* [AArch64][SVE] Implement additional floating-point arithmetic intrinsicsKerry McLaughlin2019-11-141-0/+637
| | | | | | | | | | | | | | | | | | | Summary: Adds intrinsics for the following: - ftssel - fcadd, fcmla - fmla, fmls, fnmla, fnmls - fmad, fmsb, fnmad, fnmsb Reviewers: sdesmalen, huntergr, dancgr, mgudim Reviewed By: sdesmalen Subscribers: tschuett, kristof.beyls, hiraditya, rkruppe, psnobl, cameron.mcinally, cfe-commits, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D69707
* [AArch64][v8.3a] Add missing imp-defs on RETA*.Ahmed Bougacha2019-11-131-2/+2
| | | | | | | | | | | | | RETA always implicitly uses LR, unlike RET which merely has an alias that defaults it to LR. Additionally, RETA implicitly uses SP as well, which it uses as a discriminator to authenticate LR. This isn't usually noticeable, because RET_ReallyLR is used in most of the backend. However, the post-RA scheduler, if enabled, will cause miscompiles if the imp-uses are missing. While there, fix a typo in the lone affected testcase.
* [AArch64] Extend storeRegToStackSlot to spill SVE registers.Sander de Smalen2019-11-131-0/+92
| | | | | | | | | | This patch allows the register allocator to spill SVE registers to the stack. Reviewers: ostannard, efriedma, rengolin, cameron.mcinally Reviewed By: efriedma Differential Revision: https://reviews.llvm.org/D70082
* [AArch64][SVE] Allocate locals that are scalable vectors.Sander de Smalen2019-11-132-0/+84
| | | | | | | | | | | | This patch adds a target interface to set the StackID for a given type, which allows scalable vectors (e.g. `<vscale x 16 x i8>`) to be assigned a 'sve-vec' StackID, so it is allocated in the SVE area of the stack frame. Reviewers: ostannard, efriedma, rengolin, cameron.mcinally Reviewed By: efriedma Differential Revision: https://reviews.llvm.org/D70080
* [AArch64ExpandPseudos] Preserve renamable state when expanding MOVi64 & co.Florian Hahn2019-11-121-0/+102
| | | | | | | | | | | If the MOVi operand was renamable, the operands of the expanded instructions are also renamable. Reviewers: thegameg, samparker, zatrazz Reviewed By: thegameg Differential Revision: https://reviews.llvm.org/D70061
* [CGP] Make ICMP_EQ use CR result of ICMP_S(L|G)T dominatorsYi-Hong Lyu2019-11-111-11/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For example: long long test(long long a, long long b) { if (a << b > 0) return b; if (a << b < 0) return a; return a*b; } Produces: sld. 5, 3, 4 ble 0, .LBB0_2 mr 3, 4 blr .LBB0_2: # %if.end cmpldi 5, 0 li 5, 1 isel 4, 4, 5, 2 mulld 3, 4, 3 blr But the compare (cmpldi 5, 0) is redundant and can be removed (CR0 already contains the result of that comparison). The root cause of this is that LLVM converts signed comparisons into equality comparison based on dominance. Equality comparisons are unsigned by default, so we get either a record-form or cmp (without the l for logical) feeding a cmpl. That is the situation we want to avoid here. Differential Revision: https://reviews.llvm.org/D60506
* [AArch64][SVE] Spilling/filling of SVE callee-saves.Sander de Smalen2019-11-111-0/+184
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implement the spills/fills of callee-saved SVE registers using STR and LDR instructions. Also adds the `aarch64_sve_vector_pcs` attribute to specify the callee-saved registers to be used for functions that return SVE vectors or take SVE vectors as arguments. The callee-saved registers are vector registers z8-z23 and predicate registers p4-p15. The overal frame-layout with SVE will be as follows: +-------------+ | stack args | +-------------+ | Callee Saves| | X29, X30 | |-------------| <- FP | SVE Callee | < ////////////// | saved regs | < ////////////// | z23 | < ////////////// | : | < // SCALABLE // | z8 | < ////////////// | p15 | < /// STACK //// | : | < ////////////// | p4 | < //// AREA //// +-------------+ < ////////////// | : | < ////////////// | SVE locals | < ////////////// | : | < ////////////// +-------------+ |/////////////| alignment gap. | : | | Stack objs | | : | +-------------+ <- SP after call and frame-setup Reviewers: cameron.mcinally, efriedma, greened, thegameg, ostannard, rengolin Reviewed By: ostannard Differential Revision: https://reviews.llvm.org/D68996
* [NFC] Add one test to verify the dependency brings by Macro-Fusion.QingShan Zhang2019-11-111-0/+23
|
* [NFC][Codegen] Add `x u% C1 == C2` with C1 u> C2 non-tautological testsRoman Lebedev2019-11-102-0/+365
|
* [NFC][Codegen] Add `x u% C1 == C2` with C1 u<= C2 tautological testsRoman Lebedev2019-11-101-0/+127
|
* [AArch64][X86] Don't assume __powidf2 is available on Windows.Eli Friedman2019-11-081-0/+46
| | | | | | | | | | We had some code for this for 32-bit ARM, but this doesn't really need to be in target-specific code; generalize it. (I think this started showing up recently because we added an optimization that converts pow to powi.) Differential Revision: https://reviews.llvm.org/D69013
* [RAGreedy] Enable -consider-local-interval-cost for AArch64Sanne Wouda2019-11-081-67/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The greedy register allocator occasionally decides to insert a large number of unnecessary copies, see below for an example. The -consider-local-interval-cost option (which X86 already enables by default) fixes this. We enable this option for AArch64 only after receiving feedback that this change is not beneficial for PowerPC. We evaluated the impact of this change on compile time, code size and performance benchmarks. This option has a small impact on compile time, measured on CTMark. A 0.1% geomean regression on -O1 and -O2, and 0.2% geomean for -O3, with at most 0.5% on individual benchmarks. The effect on both code size and performance on AArch64 for the LLVM test suite is nil on the geomean with individual outliers (ignoring short exec_times) between: best worst size..text -3.3% +0.0% exec_time -5.8% +2.3% On SPEC CPU® 2017 (compiled for AArch64) there is a minor reduction (-0.2% at most) in code size on some benchmarks, with a tiny movement (-0.01%) on the geomean. Neither intrate nor fprate show any change in performance. This patch makes the following changes. - For the AArch64 target, enableAdvancedRASplitCost() now returns true. - Ensures that -consider-local-interval-cost=false can disable the new behaviour if necessary. This matrix multiply example: $ cat test.c long A[8][8]; long B[8][8]; long C[8][8]; void run_test() { for (int k = 0; k < 8; k++) { for (int i = 0; i < 8; i++) { for (int j = 0; j < 8; j++) { C[i][j] += A[i][k] * B[k][j]; } } } } results in the following generated code on AArch64: $ clang --target=aarch64-arm-none-eabi -O3 -S test.c -o - [...] // %for.cond1.preheader // =>This Inner Loop Header: Depth=1 add x14, x11, x9 str q0, [sp, #16] // 16-byte Folded Spill ldr q0, [x14] mov v2.16b, v15.16b mov v15.16b, v14.16b mov v14.16b, v13.16b mov v13.16b, v12.16b mov v12.16b, v11.16b mov v11.16b, v10.16b mov v10.16b, v9.16b mov v9.16b, v8.16b mov v8.16b, v31.16b mov v31.16b, v30.16b mov v30.16b, v29.16b mov v29.16b, v28.16b mov v28.16b, v27.16b mov v27.16b, v26.16b mov v26.16b, v25.16b mov v25.16b, v24.16b mov v24.16b, v23.16b mov v23.16b, v22.16b mov v22.16b, v21.16b mov v21.16b, v20.16b mov v20.16b, v19.16b mov v19.16b, v18.16b mov v18.16b, v17.16b mov v17.16b, v16.16b mov v16.16b, v7.16b mov v7.16b, v6.16b mov v6.16b, v5.16b mov v5.16b, v4.16b mov v4.16b, v3.16b mov v3.16b, v1.16b mov x12, v0.d[1] fmov x15, d0 ldp q1, q0, [x14, #16] ldur x1, [x10, #-256] ldur x2, [x10, #-192] add x9, x9, #64 // =64 mov x13, v1.d[1] fmov x16, d1 ldr q1, [x14, #48] mul x3, x15, x1 mov x14, v0.d[1] fmov x17, d0 mov x18, v1.d[1] fmov x0, d1 mov v1.16b, v3.16b mov v3.16b, v4.16b mov v4.16b, v5.16b mov v5.16b, v6.16b mov v6.16b, v7.16b mov v7.16b, v16.16b mov v16.16b, v17.16b mov v17.16b, v18.16b mov v18.16b, v19.16b mov v19.16b, v20.16b mov v20.16b, v21.16b mov v21.16b, v22.16b mov v22.16b, v23.16b mov v23.16b, v24.16b mov v24.16b, v25.16b mov v25.16b, v26.16b mov v26.16b, v27.16b mov v27.16b, v28.16b mov v28.16b, v29.16b mov v29.16b, v30.16b mov v30.16b, v31.16b mov v31.16b, v8.16b mov v8.16b, v9.16b mov v9.16b, v10.16b mov v10.16b, v11.16b mov v11.16b, v12.16b mov v12.16b, v13.16b mov v13.16b, v14.16b mov v14.16b, v15.16b mov v15.16b, v2.16b ldr q2, [sp] // 16-byte Folded Reload fmov d0, x3 mul x3, x12, x1 [...] With -consider-local-interval-cost the same section of code results in the following: $ clang --target=aarch64-arm-none-eabi -mllvm -consider-local-interval-cost -O3 -S test.c -o - [...] .LBB0_1: // %for.cond1.preheader // =>This Inner Loop Header: Depth=1 add x14, x11, x9 ldp q0, q1, [x14] ldur x1, [x10, #-256] ldur x2, [x10, #-192] add x9, x9, #64 // =64 mov x12, v0.d[1] fmov x15, d0 mov x13, v1.d[1] fmov x16, d1 ldp q0, q1, [x14, #32] mul x3, x15, x1 cmp x9, #512 // =512 mov x14, v0.d[1] fmov x17, d0 fmov d0, x3 mul x3, x12, x1 [...] Reviewers: SjoerdMeijer, samparker, dmgreen, qcolombet Reviewed By: dmgreen Subscribers: ZhangKang, jsji, wuzish, ppc-slack, lkail, steven.zhang, MatzeB, qcolombet, kristof.beyls, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D69437
* [llvm/test] Update test commentsShu-Chun Weng2019-11-071-5/+5
|
* [AArch64][SVE] Add remaining patterns and intrinsics for add/sub/mad patternsDanilo Carvalho Grael2019-11-065-5/+832
| | | | | | | | | | | Add pattern matching and intrinsics for the following instructions: predicated orr, eor, and, bic predicated mul, smulh, umulh, sdiv, udiv, sdivr, udivr predicated smax, umax, smin, umin, sabd, uabd mad, msb, mla, mls https://reviews.llvm.org/D69588
* When lowering calls and tail calls in AArch64, the register mask andEric Christopher2019-11-063-2/+40
| | | | | | | | | | return value location depends on the calling convention of the callee. `F.getCallingConv()`, however, is the caller CC. Correct it to the callee CC from `CallLoweringInfo`. Fixes PR43449 Patch by Shu-Chun Weng!
* [AArch64] Re-add patterns for (s/u)mull2.Eli Friedman2019-11-061-0/+60
| | | | | | | These patterns were added in D46009, but removed in D54276 due to missing test coverage. Differential Revision: https://reviews.llvm.org/D69831
* [GISel][ArtifactCombiner] Relax the constraint to combine unmerge with ↵Quentin Colombet2019-11-062-11/+83
| | | | | | | | | | | | | | | concat_vectors The combine G_UNMERGE_VALUES with G_CONCAT_VECTORS used to only be performed when the result type of the G_UNMERGE_VALUES was a vector type. In other words, we were expecting that the G_UNMERGE_VALUES was effectively the exact opposite of the G_CONCAT_VECTORS. Lift that constraint by allowing any G_UNMERGE_VALUES to be combined with any G_CONCAT_VECTORS (as long as the size of the different pieces that we merge/unmerge match). Differential Revision: https://reviews.llvm.org/D69288
* [AArch64] Move the branch relaxation pass after BTI insertionMomchil Velikov2019-11-063-2/+66
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Inserting BTI instructions can push branch destinations out of range. The branch relaxation pass itself cannot insert indirect branches since `TargetInstrInfo::insertIndirecrtBranch` is not implemented for AArch64 (guess +/-128 MB direct branch range is more than enough in practice). Testing this is a bit tricky. The original test case we have is 155kloc/6.1M. I've generated a test case using this program: ``` int main() { std::cout << R"src(int test(); void g0(), g1(), g2(), g3(), g4(), e(); void f(int v) { if ((test() & 2) == 0) { switch (v) { case 0: g0(); case 1: g1(); case 2: g2(); case 3: g3(); } )src"; const int N = 8176; for (int i = 0; i < N; ++i) std::cout << " void h" << i << "();\n"; for (int i = 0; i < N; ++i) std::cout << " h" << i << "();\n"; std::cout << R"src( } else { e(); } } )src"; } ``` which is still a bit too much to commit as a regression test, IMHO. Reviewers: t.p.northover, ostannard Reviewed By: ostannard Subscribers: kristof.beyls, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D69118 Change-Id: Ide5c922bcde08ff4cf635da5e52365525a997a0a
* [globalisel] Rename G_GEP to G_PTR_ADDDaniel Sanders2019-11-0523-195/+195
| | | | | | | | | | | | | | | | Summary: G_GEP is rather poorly named. It's a simple pointer+scalar addition and doesn't support any of the complexities of getelementptr. I therefore propose that we rename it. There's a G_PTR_MASK so let's follow that convention and go with G_PTR_ADD Reviewers: volkan, aditya_nandakumar, bogner, rovka, arsenm Subscribers: sdardis, jvesely, wdng, nhaehnle, hiraditya, jrtc27, atanasyan, arphaman, Petar.Avramovic, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D69734
* [MachineScheduler] Enable AA in PostRA Machine schedulerDavid Green2019-11-051-3/+3
| | | | | | | | | | | | This adds AA to Post-RA Machine Scheduling, allowing the pass more freedom when handling memory operations. My understanding is that this was just never done, not that it is inherently incorrect to do so. The older PostRA List scheduler already makes use of AA, it's just that the MI PostRA Scheduler was never taught to use it. Differential Revision: https://reviews.llvm.org/D69814
* [AArch64] Update test checks on merge-store-dependency.ll. NFCDavid Green2019-11-051-4/+42
|
* Revert "[AArch64][MachineOutliner] Return address signing for outlined ↵Oliver Stannard2019-11-019-725/+0
| | | | | | | | | | functions" This is causing faults when an instruction which modifies SP is outlined, causing the PAC and AUT instructions to not match. This reverts commits 70caa1fc30c392974df3bccd9959765dae1779f6 and 55314d323738e4a8c1890b6a6e5064e7f4e0da1c.
* [AArch64] Output the pseudo SPACE in asm and object filesMomchil Velikov2019-11-011-0/+16
| | | | | | | | | | | | | | | | Summary: It outputs nothing, but is useful for writing tests, checking asm output. Reviewers: t.p.northover, ostannard, tellenbach Reviewed By: tellenbach Subscribers: tellenbach, kristof.beyls, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D69185 Change-Id: I6b58310e9e5632f0976d2000ce975ee28df90ebe
* [LegacyPM] Fix pass structure dumpingevgeny2019-11-011-1/+2
| | | | | | | | If module pass uses on-demand function analyses then structure is being displayed incorrectly because FunctionPassManagerImpl can't dump contained FPPassManager instances. Differential revision: https://reviews.llvm.org/D69315
* [AArch64][SVE] Implement several floating-point arithmetic intrinsicsKerry McLaughlin2019-11-011-0/+530
| | | | | | | | | | | | | | | | | | | Summary: Adds intrinsics for the following: - fabd, fadd, fsub & fsubr - fmul, fmulx, fdiv & fdivr - fmax, fmaxnm, fmin & fminnm - fscale & ftsmul Reviewers: huntergr, sdesmalen, dancgr Reviewed By: sdesmalen Subscribers: tschuett, kristof.beyls, hiraditya, rkruppe, psnobl, cameron.mcinally, cfe-commits, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D69657
OpenPOWER on IntegriCloud