summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Target
Commit message (Collapse)AuthorAgeFilesLines
...
* [X86] Standardize shuffle match/lowering function names. NFC.Simon Pilgrim2020-01-071-38/+39
| | | | We mainly use lowerShuffle*/matchShuffle* - replace the (few) lowerVectorShuffle*/matchVectorShuffle* cases to be consistent.
* [ARM] Improve codegen of volatile load/store of i64Victor Campos2020-01-076-6/+162
| | | | | | | | | | | | | | | | | | Summary: Instead of generating two i32 instructions for each load or store of a volatile i64 value (two LDRs or STRs), now emit LDRD/STRD. These improvements cover architectures implementing ARMv5TE or Thumb-2. Reviewers: dmgreen, efriedma, john.brawn, nickdesaulniers Reviewed By: efriedma, nickdesaulniers Subscribers: nickdesaulniers, vvereschaka, kristof.beyls, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D70072
* Fix "use of uninitialized variable" static analyzer warning. NFCI.Simon Pilgrim2020-01-071-1/+1
|
* Fix "use of uninitialized variable" static analyzer warnings. NFCI.Simon Pilgrim2020-01-071-2/+2
|
* [MC] Add parameter `Address` to MCInstrPrinter::printInstructionFangrui Song2020-01-0638-51/+51
| | | | | | | | Follow-up of D72172. Reviewed By: jhenderson, rnk Differential Revision: https://reviews.llvm.org/D72180
* [MC] Add parameter `Address` to MCInstPrinter::printInstFangrui Song2020-01-0640-93/+111
| | | | | | | | | | | | | | | | | | | | | | | | printInst prints a branch/call instruction as `b offset` (there are many variants on various targets) instead of `b address`. It is a convention to use address instead of offset in most external symbolizers/disassemblers. This difference makes `llvm-objdump -d` output unsatisfactory. Add `uint64_t Address` to printInst(), so that it can pass the argument to printInstruction(). `raw_ostream &OS` is moved to the last to be consistent with other print* methods. The next step is to pass `Address` to printInstruction() (generated by tablegen from the instruction set description). We can gradually migrate targets to print addresses instead of offsets. In any case, downstream projects which don't know `Address` can pass 0 as the argument. Reviewed By: jhenderson Differential Revision: https://reviews.llvm.org/D72172
* AMDGPU/GlobalISel: Fix unused variable warning in releaseMatt Arsenault2020-01-061-2/+1
|
* AMDGPU: Select llvm.amdgcn.interp.p2.f16 directlyMatt Arsenault2020-01-062-26/+12
| | | | This will enable automatic GlobalISel support in a future commit.
* AMDGPU: Use default operands for clamp/omodMatt Arsenault2020-01-061-10/+28
| | | | | | | We have a lot of complex pattern variants that just set the source modifiers that are really handled, and then set the output modifiers to 0. We're unlikely to ever match output modifiers from the use instruction side, and we already match clamp/omod in a separate pass.
* [WebAssembly] Fix landingpad-only case in Emscripten EHHeejin Ahn2020-01-061-1/+1
| | | | | | | | | | | | | | | | | | | Summary: Previously we didn't set `Changed` to true when there are only landing pads but not invokes. This fixes it and we set `Changed` to true whenever we have landing pads. (There can't be invokes without landing pads, so that case is covered too) The test case for this has to be a separate file because this pass is a `ModulePass` and `Changed` is computed based on the whole module. Reviewers: tlively Subscribers: dschuff, sbc100, jgravelle-google, hiraditya, sunfish, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D72308
* AMDGPU/GlobalISel: Legalize G_READCYCLECOUNTERMatt Arsenault2020-01-062-1/+5
|
* AMDGPU/GlobalISel: Select G_UADDE/G_USUBEMatt Arsenault2020-01-062-11/+31
|
* AMDGPU/GlobalISel: Replace handling of boolean valuesMatt Arsenault2020-01-068-249/+427
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This solves selection failures with generated selection patterns, which would fail due to inferring the SGPR reg bank for virtual registers with a set register class instead of VCC bank. Use instruction selection would constrain the virtual register to a specific class, so when the def was selected later the bank no longer was set to VCC. Remove the SCC reg bank. SCC isn't directly addressable, so it requires copying from SCC to an allocatable 32-bit register during selection, so these might as well be treated as 32-bit SGPR values. Now any scalar boolean value that will produce an outupt in SCC should be widened during RegBankSelect to s32. Any s1 value should be a vector boolean during selection. This makes the vcc register bank unambiguous with a normal SGPR during selection. Summary of how this should now work: - G_TRUNC is always a no-op, and never should use a vcc bank result. - SALU boolean operations should be promoted to s32 in RegBankSelect apply mapping - An s1 value means vcc bank at selection. The exception is for legalization artifacts that use s1, which are never VCC. All other contexts should infer the VCC register classes for s1 typed registers. The LLT for the register is now needed to infer the correct register class. Extensions with vcc sources should be legalized to a select of constants during RegBankSelect. - Copy from non-vcc to vcc ensures high bits of the input value are cleared during selection. - SALU boolean inputs should ensure the inputs are 0/1. This includes select, conditional branches, and carry-ins. There are a few somewhat dirty details. One is that G_TRUNC/G_*EXT selection ignores the usual register-bank from register class functions, and can't handle truncates with VCC result banks. I think this is OK, since the artifacts are specially treated anyway. This does require some care to avoid producing cases with vcc. There will also be no 100% reliable way to verify this rule is followed in selection in case of register classes, and violations manifests themselves as invalid copy instructions much later. Standard phi handling also only considers the bank of the result register, and doesn't insert copies to make the source banks match. This doesn't work for vcc, so we have to manually correct phi inputs in this case. We should add a verifier check to make sure there are no phis with mixed vcc and non-vcc register bank inputs. There's also some duplication with the LegalizerHelper, and some code which should live in the helper. I don't see a good way to share special knowledge about what types to use for intermediate operations depending on the bank for example. Using the helper to replace extensions with selects also seems somewhat awkward to me. Another issue is there are some contexts calling getRegBankFromRegClass that apparently don't have the LLT type for the register, but I haven't yet run into a real issue from this. This also introduces new unnecessary instructions in most cases, since we don't yet try to optimize out the zext when the source is known to come from a compare.
* GlobalISel: Implement lower for G_INTRINSIC_ROUNDMatt Arsenault2020-01-062-3/+2
| | | | | Mostly copied from AMDGPU lowering implementation, except used G_SITOFP instead of directly creating a select on -1.0, 0.0.
* [X86] Move an enum definition into a header to simplify future patches [NFC]Philip Reames2020-01-062-24/+26
|
* [PowerPC][NFC] Rename record instructions to use _rec suffix instead of oJinsong Ji2020-01-0613-1007/+1050
| | | | | | | | | | | | | | | | | | | We use o suffix to indicate record form instuctions, (as it is similar to dot '.' in mne?) This was fine before, as we did not support XO-form. However, with https://reviews.llvm.org/D66902, we now have XO-form support. It becomes confusing now to still use 'o' for record form, and it is weird to have something like 'Oo' . This patch rename all 'o' instructions to use '_rec' instead. Also rename `isDot` to `isRecordForm`. Reviewed By: #powerpc, hfinkel, nemanjai, steven.zhang, lkail Differential Revision: https://reviews.llvm.org/D70758
* GlobalISel: Fix unsupported legalize actionMatt Arsenault2020-01-061-0/+5
| | | | | | | | This would complain about invalid legalizer rules otherwise. Mark some operations as unsupported for AMDGPU. This currently seems to produce the same legalize error as when no rules are defined, but eventually this should produce a proper user facing error.
* AMDGPU: Fix legalizing f16 fpowMatt Arsenault2020-01-062-0/+2
| | | | | | The existing test only covered one case for r600. The use of mul_legacy also looks suspicious to me, but leave it for now. The patterns are also not making use of source modifiers.
* AMDGPU: Use ImmLeafMatt Arsenault2020-01-061-2/+2
| | | | | This solves one GlobalISel importer error, but the pattern still fails for another reason.
* AMDGPU: Use ImmLeaf for inline immediate predicatesMatt Arsenault2020-01-066-9/+63
|
* [X86] Improve v4i32->v4f64 uint_to_fp for AVX1/AVX2 targets.Craig Topper2020-01-061-0/+15
| | | | | | Use zext+or+fsub to do the conversion. Similar to D71971. Differential Revision: https://reviews.llvm.org/D71971
* Lower TAGPstack with negative offset to SUBG.Evgenii Stepanov2020-01-062-3/+12
| | | | | | | | | | | | | | Summary: This never really occurs in the current codegen, so only a MIR test is possible. Reviewers: ostannard, pcc Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D72123
* [X86] Fix an 8 bit testb being selected when folding a volatile i32 load ↵Amara Emerson2020-01-061-0/+11
| | | | | | pattern. Differential Revision: https://reviews.llvm.org/D71581
* [PowerPC][LoopVectorize] Extend getRegisterClassForType to consider double ↵Jinsong Ji2020-01-061-3/+9
| | | | | | | | | | | | | | and other floating point type In https://reviews.llvm.org/D67148, we use isFloatTy to test floating point type, otherwise we return GPRRC. So 'double' will be classified as GPRRC, which is not accurate. This patch covers other floating point types. Reviewed By: #powerpc, nemanjai Differential Revision: https://reviews.llvm.org/D71946
* [AIX] Use csect reference for function address constantsdiggerlin2020-01-061-1/+1
| | | | | | | | | | | | | | | | | | | | SUMMARY: We currently emit a reference for function address constants as labels; for example: foo_ptr: .long foo however, there may be no such label in the case where the function is undefined. Although the label exists when the function is defined, we will (to be consistent) also use a csect reference in that case. Address one comment https://reviews.llvm.org/D71144#inline-653255 Reviewers: daltenty,hubert.reinterpretcast,jasonliu,Xiangling_L Subscribers: cebowleratibm, wuzish, nemanjai Differential Revision: https://reviews.llvm.org/D71144
* [ARM] Use the correct opcodes for Thumb2 segmented stack frame loweringDavid Green2020-01-061-2/+4
| | | | | | | | | The segmented stack lowering code appears to be using ARM opcodes under Thumb2. The MRC opcode will be the same for Thumb and ARM, but t2LDR seems wrong. Either way, using the correct thumb vs arm opcodes is more correct. Differential Revision: https://reviews.llvm.org/D72074
* [ARM] Use correct TRAP opcode for thumb in FastISelDavid Green2020-01-061-2/+6
| | | | | | | | | We were previously unconditionally using the ARM::TRAP opcode, even under Thumb. My understanding is that these are essentially the same thing (they both result in a trap under Thumb), but the ARM::TRAP opcode is marked as requiring IsARM, so it is more correct to use ARM::tTRAP. Differential Revision: https://reviews.llvm.org/D72075
* [AIX] Use csect reference for function address constantsdiggerlin2020-01-061-0/+22
| | | | | | | | | | | | | | | | | SUMMARY: We currently emit a reference for function address constants as labels; for example: foo_ptr: .long foo however, there may be no such label in the case where the function is undefined. Although the label exists when the function is defined, we will (to be consistent) also use a csect reference in that case. Reviewers: daltenty,hubert.reinterpretcast,jasonliu,Xiangling_L Subscribers: cebowleratibm, wuzish, nemanjai Differential Revision: https://reviews.llvm.org/D71144
* [AMDGPU] Fix "use of uninitialized variable" static analyzer warning. NFCI.Simon Pilgrim2020-01-061-0/+1
| | | | Add "unreachable" default case to AMDGPUTargetStreamer::getArchNameFromElfMach
* Fix "use of uninitialized variable" static analyzer warnings. NFCI.Simon Pilgrim2020-01-061-0/+2
| | | | Add "unreachable" default cases like we do for the other switch()s in X86MCInstLower::Lower
* [ARM,MVE] Fix many signedness errors in MVE intrinsics.Simon Tatham2020-01-061-28/+49
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Running an end-to-end test last week I noticed that a lot of the ACLE intrinsics that operate differently on vectors of signed and unsigned integers were ending up generating the signed version of the instruction unconditionally. This is because the IR intrinsics had no way to distinguish signed from unsigned: the LLVM type system just calls them both `v8i16` (or whatever), so you need either separate intrinsics for signed and unsigned, or a flag parameter that tells ISel which one to choose. This patch fixes all the problems of that kind that I've noticed, by adding an i32 flag parameter to many of the IR intrinsics which is set to 1 for unsigned (matching the existing practice in cases where we got it right), and conditioning all the isel patterns on that flag. So the fundamental change is in `IntrinsicsARM.td`, changing the low-level IR intrinsics API; there are knock-on changes in `arm_mve.td` (adjusting code gen for the ACLE intrinsics to use the modified API) and in `ARMInstrMVE.td` (adjusting isel to expect the new unsigned flags). The rest of this patch is boringly updating tests. Reviewers: dmgreen, miyuki, MarkMurrayARM Reviewed By: dmgreen Subscribers: kristof.beyls, hiraditya, cfe-commits, llvm-commits Tags: #clang, #llvm Differential Revision: https://reviews.llvm.org/D72270
* [ARM,MVE] Generate the right instruction for vmaxnmq_m_f16.Simon Tatham2020-01-061-2/+2
| | | | | | | | | | | | | | | | | Summary: Due to a copy-paste error in the isel patterns, the predicated version of this intrinsic was expanding to the `VMAXNMT.F32` instruction instead of `VMAXNMT.F16`. Similarly for vminnm. Reviewers: dmgreen, miyuki, MarkMurrayARM Reviewed By: dmgreen Subscribers: kristof.beyls, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D72269
* AMDGPU/GlobalISel: Select scalar v2s16 G_BUILD_VECTORMatt Arsenault2020-01-063-25/+40
|
* AMDGPU/GlobalISel: Select more G_EXTRACTs correctlyMatt Arsenault2020-01-061-5/+19
| | | | | | | | This assumed a 32-bit extract size, which would produce invalid copies with 64-bit extracts. Handle the easy case. Ideally we would have a way to get the proper subreg index for any 32-bit offset, but there should probably be a tablegenerated way of getting the subreg index for any size and offset.
* [CostModel][X86] Add missing scalar i64->f32 uitofp costsSimon Pilgrim2020-01-061-0/+4
|
* [NFC] Fix trivial typos in commentsJames Henderson2020-01-0613-13/+13
| | | | | | | | Reviewed By: jhenderson Differential Revision: https://reviews.llvm.org/D72143 Patch by Kazuaki Ishizaki.
* [ARM][MVE] More MVETailPredication debug messages. NFC.Sjoerd Meijer2020-01-062-65/+96
| | | | | | | | | | I've added a few more debug messages to MVETailPredication because I wanted to trace better which instructions are added/removed. And while I was at it, I factored out one function which I thought was clearer, and have added some comments to describe better the flow between MVETailPredication and ARMLowOverheadLoops. Differential Revision: https://reviews.llvm.org/D71549
* Add interface emitPrefix for MCCodeEmitterShengchen Kan2020-01-061-89/+133
| | | | Differential Revision: https://reviews.llvm.org/D72047
* [APFloat] Add recoverable string parsing errors to APFloatEhud Katz2020-01-062-6/+12
| | | | | | Implementing the APFloat part in PR4745. Differential Revision: https://reviews.llvm.org/D69770
* [X86] Improve v2i64->v2f32 and v4i64->v4f32 uint_to_fp on avx and avx2 targets.Craig Topper2020-01-051-24/+125
| | | | | | | | | | | | | | | | | | | | | Summary: Based on Simon's D52965, but improved to handle strict fp and improve some of the shuffling. Rather than use v2i1/v4i1 and let type legalization continue, just generate all the code with legal types and use an explicit shuffle. I also added an explicit setcc to the v4i64 code to match the semantics of vselect which doesn't just use the sign bit. I'm also using a v4i64->v4i32 truncate instead of the shuffle in Simon's original code. With the setcc this will become a pack. Future work can look into using X86ISD::BLENDV and a different shuffle that only moves the sign bit. Reviewers: RKSimon, spatel Reviewed By: RKSimon Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71956
* [NFC] Modify the format:Liu, Chen32020-01-061-2/+1
| | | | Drop the else since we alerady returned in the if.
* [MC][ARM] Delete MCSection::HasData and move SHF_ARM_PURECODE logic to ↵Fangrui Song2020-01-051-2/+5
| | | | | | | | ARMELFObjectWriter::addTargetSectionFlags This simplifies the generic interface and also makes SHF_ARM_PURECODE more robust (fixes a TODO). Inspecting MCDataFragment contents covers more cases than MCObjectStreamer::EmitBytes.
* [X86][SSE] Combine combineLogicBlendIntoConditionalNegate for VSELECT nodes ↵Simon Pilgrim2020-01-051-2/+13
| | | | | | | | (PR43660) Attempt to use combineLogicBlendIntoConditionalNegate for (select M, (sub 0, X), X) -> (sub (xor X, M), M) We limit this to cases that can't easily replace the VSELECT with a shuffle (non-constant masks) or where a BLENDV is likely to occur (which tends to result in slower codegen).
* [X86] Move combineLogicBlendIntoConditionalNegate before combineSelect. NFCI.Simon Pilgrim2020-01-051-62/+62
| | | | Updates function order in preparation of future fix for PR43660
* [X86] Merge (identical) LowerGC_TRANSITION_START and LowerGC_TRANSITION_END ↵Simon Pilgrim2020-01-052-27/+4
| | | | | | (NFC) Silences a copy+paste analyzer warning - all they are doing are inserting NOOPs in exactly the same way.
* [ARM] Use isFMAFasterThanFMulAndFAdd for scalars as well as MVE vectorsDavid Green2020-01-055-18/+45
| | | | | | | | | | | This adds extra scalar handling to isFMAFasterThanFMulAndFAdd, allowing the target independent code to handle more folds in more situations (for example if the fast math flags are present, but the global AllowFPOpFusion option isnt). It also splits apart the HasSlowFPVMLx into HasSlowFPVFMx, to allow VFMA and VMLA to be controlled separately if needed. Differential Revision: https://reviews.llvm.org/D72139
* [ARM] Fill in FP16 FMA patternsDavid Green2020-01-051-0/+21
| | | | | | This adds fp16 variants of all the fma patterns in the ARM backend. Differential Revision: https://reviews.llvm.org/D72138
* GlobalISel: Scalarize all division operationsMatt Arsenault2020-01-041-0/+7
| | | | | | This only handled G_SDIV, but they all are trivially scalarizable. Also define placeholder AMDGPU division legalizer rules.
* Revert "[SCEV] Move ScalarEvolutionExpander.cpp to Transforms/Utils (NFC)."Florian Hahn2020-01-043-3/+3
| | | | | This reverts commit 51ef53f3bd23559203fe9af82ff2facbfedc1db3, as it breaks some bots.
* [SCEV] Move ScalarEvolutionExpander.cpp to Transforms/Utils (NFC).Florian Hahn2020-01-043-3/+3
| | | | | | | | | | | | SCEVExpander modifies the underlying function so it is more suitable in Transforms/Utils, rather than Analysis. This allows using other transform utils in SCEVExpander. Reviewers: sanjoy.google, efriedma, reames Reviewed By: sanjoy.google Differential Revision: https://reviews.llvm.org/D71537
OpenPOWER on IntegriCloud