summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Target/AArch64
Commit message (Collapse)AuthorAgeFilesLines
* [AArch64] Swap comparison operands if that enables some folding.Arnaud A. de Grandmaison2018-10-131-12/+74
| | | | | | | | | | | | | | | | Summary: AArch64 can fold some shift+extend operations on the RHS operand of comparisons, so swap the operands if that makes sense. This provides a fix for https://bugs.llvm.org/show_bug.cgi?id=38751 Reviewers: efriedma, t.p.northover, javed.absar Subscribers: mcrosier, kristof.beyls, llvm-commits Differential Revision: https://reviews.llvm.org/D53067 llvm-svn: 344439
* [AARCH64][FIX] Emit data symbol for constant pool dataDiogo N. Sampaio2018-10-111-0/+5
| | | | | | | | | | | The ARM64 elf emitter would omit printing data symbol for zero filled constant data. This patch overrides the emitFill method as to enforce that the symbol is correctly printed. Differential revision: https://reviews.llvm.org/D53132 llvm-svn: 344248
* [AArch64][v8.5A] Don't create BR instructions in outliner when BTI enabledOliver Stannard2018-10-081-1/+9
| | | | | | | | | | When branch target identification is enabled, we can only do indirect tail-calls through x16 or x17. This means that the outliner can't transform a BLR instruction at the end of an outlined region into a BR. Differential revision: https://reviews.llvm.org/D52869 llvm-svn: 343969
* [AArch64][v8.5A] Restrict indirect tail calls to use x16/17 only when using BTIOliver Stannard2018-10-084-2/+21
| | | | | | | | | | | | | | | | | | | | | | | When branch target identification is enabled, all indirectly-callable functions start with a BTI C instruction. this instruction can only be the target of certain indirect branches (direct branches and fall-through are not affected): - A BLR instruction, in either a protected or unprotected page. - A BR instruction in a protected page, using x16 or x17. - A BR instruction in an unprotected page, using any register. Without BTI, we can use any non call-preserved register to hold the address for an indirect tail call. However, when BTI is enabled, then the code being compiled might be loaded into a BTI-protected page, where only x16 and x17 can be used for indirect tail calls. Legacy code withiout this restriction can still indirectly tail-call BTI-protected functions, because they will be loaded into an unprotected page, so any register is allowed. Differential revision: https://reviews.llvm.org/D52868 llvm-svn: 343968
* [AArch64][v8.5A] Branch Target Identification code-generation passOliver Stannard2018-10-084-0/+142
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The Branch Target Identification extension, introduced to AArch64 in Armv8.5-A, adds the BTI instruction, which is used to mark valid targets of indirect branches. When enabled, the processor will trap if an instruction in a protected page tries to perform an indirect branch to any instruction other than a BTI. The BTI instruction uses encodings which were NOPs in earlier versions of the architecture, so BTI-enabled code will still run on earlier hardware, just without the extra protection. There are 3 variants of the BTI instruction, which are valid targets for different kinds or branches: - BTI C can be targeted by call instructions, and is inteneded to be used at function entry points. These are the BLR instruction, as well as BR with x16 or x17. These BR instructions are allowed for use in PLT entries, and we can also use them to allow indirect tail-calls. - BTI J can be targeted by BR only, and is intended to be used by jump tables. - BTI JC acts ab both a BTI C and a BTI J instruction, and can be targeted by any BLR or BR instruction. Note that RET instructions are not restricted by branch target identification, the reason for this is that return addresses can be protected more effectively using return address signing. Direct branches and calls are also unaffected, as it is assumed that an attacker cannot modify executable pages (if they could, they wouldn't need to do a ROP/JOP attack). This patch adds a MachineFunctionPass which: - Adds a BTI C at the start of every function which could be indirectly called (either because it is address-taken, or externally visible so could be address-taken in another translation unit). - Adds a BTI J at the start of every basic block which could be indirectly branched to. This could be either done by a jump table, or by taking the address of the block (e.g. the using GCC label values extension). We only need to use BTI JC when a function is indirectly-callable, and takes the address of the entry block. I've not been able to trigger this from C or IR, but I've included a MIR test just in case. Using BTI C at function entries relies on the fact that no other code in BTI-protected pages uses indirect tail-calls, unless they use x16 or x17 to hold the address. I'll add that code-generation restriction as a separate patch. Differential revision: https://reviews.llvm.org/D52867 llvm-svn: 343967
* [AArch64] Fix verifier error when outlining indirect callsOliver Stannard2018-10-083-2/+9
| | | | | | | | | | | | | | | | | | The MachineOutliner for AArch64 transforms indirect calls into indirect tail calls, replacing the call with the TCRETURNri pseudo-instruction. This pseudo lowers to a BR, but has the isCall and isReturn flags set. The problem is that TCRETURNri takes a tcGPR64 as the register argument, to prevent indiret tail-calls from using caller-saved registers. The indirect calls transformed by the outliner could use caller-saved registers. This is fine, because the outliner ensures that the register is available at all call sites. However, this causes a verifier failure when the register is not in tcGPR64. The fix is to add a new pseudo-instruction like TCRETURNri, but which accepts any GPR. Differential revision: https://reviews.llvm.org/D52829 llvm-svn: 343959
* X86, AArch64, ARM: Do not attach debug location to spill/reload instructionsMatthias Braun2018-10-051-8/+2
| | | | | | | | | | | | | | This rebases and recommits r343520. hwasan should be fixed now and this shouldn't break the tests anymore. Spill/reload instructions are artificially generated by the compiler and have no relation to the original source code. So the best thing to do is not attach any debug location to them (instead of just taking the next debug location we find on following instructions). Differential Revision: https://reviews.llvm.org/D52125 llvm-svn: 343895
* [TargetRegisterInfo] Remove temporary hook enableMultipleCopyHints()Jonas Paulsson2018-10-051-2/+0
| | | | | | | | | | | | Finally all targets are enabling multiple regalloc hints, so the hook to disable this can now be removed. NFC. Review: Simon Pilgrim https://reviews.llvm.org/D52316 llvm-svn: 343851
* AArch64: Fix XSeqPairs/WSeqPairs problemsMatthias Braun2018-10-041-18/+68
| | | | | | | | | | - Fix spill/reloads of XSeqPairs failing with vregs (only physregs worked correctly) - Add missing spill/reload code for WSeqPairs class Differential Revision: https://reviews.llvm.org/D52761 llvm-svn: 343799
* Add the missing new files from r343654Daniel Sanders2018-10-031-0/+108
| | | | llvm-svn: 343655
* Re-commit: [globalisel] Add a combiner helpers for extending loads and use ↵Daniel Sanders2018-10-033-0/+9
| | | | | | | | | | | | | | | | | | | | them in a pre-legalize combiner for AArch64 Summary: Depends on D45541 Reviewers: ab, aditya_nandakumar, bogner, rtereshin, volkan, rovka, javed.absar, aemerson Subscribers: aemerson, rengolin, mgorny, javed.absar, kristof.beyls, llvm-commits Differential Revision: https://reviews.llvm.org/D45543 The previous commit failed portions of the test-suite on GreenDragon due to duplicate COPY instructions and iterator invalidation. Both issues have now been fixed. To assist with this, a helper (cloneVirtualRegister) has been added to MachineRegisterInfo that can be used to get another register that has the same type and class/bank as an existing one. llvm-svn: 343654
* Revert "X86, AArch64, ARM: Do not attach debug location to spill/reload ↵Matt Morehouse2018-10-021-4/+10
| | | | | | | | instructions" This reverts r343520 due to breakage of HWASan tests on Android. llvm-svn: 343616
* [AArch64][v8.5A] Add Memory Tagging instructionsOliver Stannard2018-10-025-22/+218
| | | | | | | | | | | This adds new instructions to manipluate tagged pointers, and to load and store the tags associated with memory. Patch by Pablo Barrio, David Spickett and Oliver Stannard! Differential revision: https://reviews.llvm.org/D52490 llvm-svn: 343572
* [AArch64][v8.5A] Add Memory Tagging system registersOliver Stannard2018-10-021-0/+16
| | | | | | | | | | | This adds new system registers introduced by the Memory Tagging extension. Patch by Pablo Barrio! Differential revision: https://reviews.llvm.org/D52488 llvm-svn: 343571
* [AArch64][v8.5A] Add MTE system instructionsOliver Stannard2018-10-022-0/+22
| | | | | | | | | | | The Memory Tagging Extension adds system instructions for data cache maintenance, implemented as new operands to the DC instruction. Patch by Pablo Barrio! Differential revision: https://reviews.llvm.org/D52487 llvm-svn: 343570
* [AArch64][v8.5A] Add MTE as an optional AArch64 extensionOliver Stannard2018-10-022-0/+5
| | | | | | | | | | | | This adds the memory tagging extension, which is an optional extension introduced in v8.5A. The new instructions and registers will be added by subsequent patches. Patch by Pablo Barrio! Differential revision: https://reviews.llvm.org/D52486 llvm-svn: 343563
* Revert: r343521 and r343541: [globalisel] Add a combiner helpers for ↵Daniel Sanders2018-10-014-117/+0
| | | | | | | | | | extending loads and use them in a pre-legalize combiner for AArch64 There's a strange assertion on two of the Green Dragon bots that goes away when this is reverted. The assertion is in RegBankAlloc and if it is this commit then -verify-machine-instrs should have caught it earlier in the pipeline. llvm-svn: 343546
* [globalisel] Add a combiner helpers for extending loads and use them in a ↵Daniel Sanders2018-10-014-0/+117
| | | | | | | | | | | | | | pre-legalize combiner for AArch64 Summary: Depends on D45541 Reviewers: ab, aditya_nandakumar, bogner, rtereshin, volkan, rovka, javed.absar, aemerson Subscribers: aemerson, rengolin, mgorny, javed.absar, kristof.beyls, llvm-commits Differential Revision: https://reviews.llvm.org/D45543 llvm-svn: 343521
* X86, AArch64, ARM: Do not attach debug location to spill/reload instructionsMatthias Braun2018-10-011-10/+4
| | | | | | | | | | | Spill/reload instructions are artificially generated by the compiler and have no relation to the original source code. So the best thing to do is not attach any debug location to them (instead of just taking the next debug location we find on following instructions). Differential Revision: https://reviews.llvm.org/D52125 llvm-svn: 343520
* [AArch64] Refactor cheap cost modelEvandro Menezes2018-10-011-12/+23
| | | | | | | Refactor the order in `TII::isAsCheapAsAMove()` to ease future development and maintenance. Practically NFC. llvm-svn: 343489
* [AArch64] Split zero cycle feature more granularlyEvandro Menezes2018-09-284-12/+20
| | | | | | | | | | Split the `zcz` feature into specific ones got GP and FP registers, `zcz-gp` and `zcz-fp`, respectively, while retaining the original feature option to mean both. Differential revision: https://reviews.llvm.org/D52621 llvm-svn: 343354
* Revert r343317Luke Cheeseman2018-09-282-16/+0
| | | | | | - asan buildbots are breaking and I need to investigate the issue llvm-svn: 343341
* Reapply changes reverted by r343235Luke Cheeseman2018-09-282-0/+16
| | | | | | | | - Add fix so that all code paths that create DWARFContext with an ObjectFile initialise the target architecture in the context - Add an assert that the Arch is known in the Dwarf CallFrameString method llvm-svn: 343317
* Remove extra whitespace. NFC. (test commit)David Spickett2018-09-281-1/+1
| | | | llvm-svn: 343301
* Revert r343192 as an ubsan build is currently failingLuke Cheeseman2018-09-272-16/+0
| | | | llvm-svn: 343235
* [AArch64] Refactor immediate details out of add/sub tblgen class (NFCI)Oliver Stannard2018-09-271-13/+19
| | | | | | | | | | | | | | Bits [23-22] are used in Add and Sub to specify the shift. The value of the shift field must be 0x; values of 1x are unallocated. MTE adds some instructions that use such encodings, and this patch refactors the Add/Sub class so that another class could derive from this one to implement other encodings and other formats of bitfields. Patch by Pablo Barrio! Differential revision: https://reviews.llvm.org/D52489 llvm-svn: 343231
* [AArch64][v8.5A] Add speculation barriers SSBB and PSSBBOliver Stannard2018-09-271-0/+2
| | | | | | | | | | | This adds two new barrier instructions which can be used to restrict speculative execution of load instructions. Patch by Pablo Barrio! Differential revision: https://reviews.llvm.org/D52483 llvm-svn: 343229
* [AArch64][v8.5A] Add Branch Target Identification instructionsOliver Stannard2018-09-2710-1/+139
| | | | | | | | | | | | This adds new instructions used by the Branch Target Identification feature. When this is enabled, these are the only instructions which can be targeted by indirect branch instructions. Patch by Pablo Barrio! Differential revision: https://reviews.llvm.org/D52485 llvm-svn: 343225
* [AArch64][v8.5A] Add speculation restriction system registersOliver Stannard2018-09-276-5/+32
| | | | | | | | | | | This adds some new system registers which can be used to restrict certain types of speculative execution. Patch by Pablo Barrio and David Spickett! Differential revision: https://reviews.llvm.org/D52482 llvm-svn: 343218
* [AArch64][v8.5A] Add Armv8.5-A random number instructionsOliver Stannard2018-09-273-0/+12
| | | | | | | | | | | | | This adds two new system registers, used to generate random numbers. This is an optional extension to v8.5-A, and will be controlled by the "+rng" modifier of the -march= and -mcpu= options. Patch by Pablo Barrio! Differential revision: https://reviews.llvm.org/D52481 llvm-svn: 343217
* [AArch64][v8.5A] Add Armv8.5-A "DC CVADP" instructionOliver Stannard2018-09-276-2/+13
| | | | | | | | | | | This adds a new variant of the DC system instruction for persistent memory. Patch by Pablo Barrio! Differential revision: https://reviews.llvm.org/D52480 llvm-svn: 343216
* [AArch64][v8.5A] Add prediction invalidation instructions to AArch64Oliver Stannard2018-09-278-4/+89
| | | | | | | | | | | | This adds new system instructions which act as barriers to speculative execution based on earlier execution within a particular execution context. Patch by Pablo Barrio! Differential revision: https://reviews.llvm.org/D52479 llvm-svn: 343214
* [AArch64][v8.5A] Add speculation barrier to AArch64 instruction setOliver Stannard2018-09-273-2/+17
| | | | | | | | | | | This is a new barrier which limits speculative execution of the instructions following it. Patch by Pablo Barrio! Differential revision: https://reviews.llvm.org/D52476 llvm-svn: 343211
* [AArch64][v8.5A] Add FRINT[32,64][Z,X] instructionsOliver Stannard2018-09-274-3/+63
| | | | | | | | | | | | These are some new variants of the "Floating-point Round to Integral" family of instructions, which round to the nearest floating-point value which fits in a 32- or 64-bit integer. Patch by Pablo Barrio! Differential revision: https://reviews.llvm.org/D52475 llvm-svn: 343209
* Reapply changes reverted in r343114, lldb patch to follow shortlyLuke Cheeseman2018-09-272-0/+16
| | | | llvm-svn: 343192
* [AArch64][v8.5A] Add PSTATE manipulation instructions XAFlag and AXFlagOliver Stannard2018-09-274-3/+35
| | | | | | | | | | | These new instructions manipluate the NZCV bits, to convert between the regular Arm floating-point comare format and an alternative format. Patch by Pablo Barrio! Differential revision: https://reviews.llvm.org/D52473 llvm-svn: 343187
* llvm::sort(C.begin(), C.end(), ...) -> llvm::sort(C, ...)Fangrui Song2018-09-271-15/+14
| | | | | | | | | | | | Summary: The convenience wrapper in STLExtras is available since rL342102. Reviewers: dblaikie, javed.absar, JDevlieghere, andreadb Subscribers: MatzeB, sanjoy, arsenm, dschuff, mehdi_amini, sdardis, nemanjai, jvesely, nhaehnle, sbc100, jgravelle-google, eraman, aheejin, kbarton, JDevlieghere, javed.absar, gbedwell, jrtc27, mgrang, atanasyan, steven_wu, george.burgess.iv, dexonsmith, kristina, jsji, llvm-commits Differential Revision: https://reviews.llvm.org/D52573 llvm-svn: 343163
* [AArch64] Extend single-operand FP insns to match Arm ARM (NFCI)Oliver Stannard2018-09-261-6/+7
| | | | | | | | | | | | | | | | | | The Armv8.3-A reference manual defines floating-point data-processing instructions with one source operand to have an opcode of 6 bits [20:15]. The current class in tablegen, BaseSingleOperandFPData, only allows [18:15]. This was ok because [20:19] could only be '00', with other encodings unallocated. Armv8.5-A brings in the FRINT group of instructions which use other values for these bits. This patch refactors the existing class a bit to allow using the full 6 bits of the opcode, as defined in the Arm ARM. Patch by Pablo Barrio! Differential revision: https://reviews.llvm.org/D52474 llvm-svn: 343120
* Revert r343112 as CallFrameString API change has broken lldb buildsLuke Cheeseman2018-09-262-16/+0
| | | | llvm-svn: 343114
* [AArch64] Refactor instructions that write PSTATE (NFCI)Oliver Stannard2018-09-261-9/+15
| | | | | | | | | | | Reuse some code in preparation for the v8.5A XAFlag/AXFlag instructions, which shares part of the encoding of the MSR-immediate. Patch by Pablo Barrio! Differential revision: https://reviews.llvm.org/D52472 llvm-svn: 343113
* [AArch64] - Return address signing dwarf supportLuke Cheeseman2018-09-262-0/+16
| | | | | | - Reapply r343089 with a fix for DebugInfo/Sparc/gnu-window-save.ll llvm-svn: 343112
* [AArch64][AsmParser] Show name of missing feature for system instructionsOliver Stannard2018-09-261-24/+35
| | | | | | | | | | | | | | | | | Parsing of the system instructions (IC, DC, AT and TLBI) uses this function to show the required architecture when the operand is valid, but the architecture is not enabled. Armv8.5A adds a few different system instructions as part of optional features, so we need to extend it to show individual features, not just base architectures. This is NFC for now, but will be used by three different features added in v8.5A, and will be tested by them. Patch by David Spickett! Differential revision: https://reviews.llvm.org/D52478 llvm-svn: 343109
* Revert r343089 "[AArch64] - Return address signing dwarf support"Hans Wennborg2018-09-262-16/+0
| | | | | | | | | | | | | | | | | | | This caused the DebugInfo/Sparc/gnu-window-save.ll test to fail. > Functions that have signed return addresses need additional dwarf support: > - After signing the LR, and before authenticating it, the LR register is in a > state the is unusable by a debugger or unwinder > - To account for this a new directive, .cfi_negate_ra_state, is added > - This directive says the signed state of the LR register has now changed, > i.e. unsigned -> signed or signed -> unsigned > - This directive has the same CFA code as the SPARC directive GNU_window_save > (0x2d), adding a macro to account for multiply defined codes > - This patch matches the gcc implementation of this support: > https://patchwork.ozlabs.org/patch/800271/ > > Differential Revision: https://reviews.llvm.org/D50136 llvm-svn: 343103
* [ARM/AArch64][v8.5A] Add Armv8.5-A targetOliver Stannard2018-09-264-0/+9
| | | | | | | | | | | | | This patch allows targeting Armv8.5-A, adding the architecture to tablegen and setting the options to be identical to Armv8.4-A for the time being. Subsequent patches will add support for the different features included in the Armv8.5-A Reference Manual. Patch by Pablo Barrio! Differential revision: https://reviews.llvm.org/D52470 llvm-svn: 343102
* [AArch64] - Return address signing dwarf supportLuke Cheeseman2018-09-262-0/+16
| | | | | | | | | | | | | | | | | Functions that have signed return addresses need additional dwarf support: - After signing the LR, and before authenticating it, the LR register is in a state the is unusable by a debugger or unwinder - To account for this a new directive, .cfi_negate_ra_state, is added - This directive says the signed state of the LR register has now changed, i.e. unsigned -> signed or signed -> unsigned - This directive has the same CFA code as the SPARC directive GNU_window_save (0x2d), adding a macro to account for multiply defined codes - This patch matches the gcc implementation of this support: https://patchwork.ozlabs.org/patch/800271/ Differential Revision: https://reviews.llvm.org/D50136 llvm-svn: 343089
* [AArch64] Share search bookkeeping in combines. NFCI.Nirav Dave2018-09-251-15/+17
| | | | | | | | Share predecessor search bookkeeping in both perform PostLD1Combine and performNEONPostLDSTCombine. This should be approximately a 4x and 2x performance improvement. llvm-svn: 342986
* [Aarch64] Fix memcpy that was copying 4x too many bytesBenjamin Kramer2018-09-231-1/+1
| | | | | | Found by asan. llvm-svn: 342845
* [AArch64] Support adding X[8-15,18] registers as CSRs.Tri Vo2018-09-229-7/+84
| | | | | | | | | | | | | | | | | | | Summary: Specifying X[8-15,18] registers as callee-saved is used to support CONFIG_ARM64_LSE_ATOMICS in Linux kernel. As part of this patch we: - use custom CSR list/mask when user specifies custom CSRs - update Machine Register Info's list of CSRs with additional custom CSRs in LowerCall and LowerFormalArguments. Reviewers: srhines, nickdesaulniers, efriedma, javed.absar Reviewed By: nickdesaulniers Subscribers: kristof.beyls, jfb, llvm-commits Differential Revision: https://reviews.llvm.org/D52216 llvm-svn: 342824
* AArch64FastISel: Abort if we failed to select operand of intrinsicMatthias Braun2018-09-211-0/+3
| | | | | | | | rdar://44642447 Differential Revision: https://reviews.llvm.org/D52335 llvm-svn: 342742
* AArch64: Add FuseCryptoEOR fusion rulesMatthias Braun2018-09-193-0/+27
| | | | | | | | There's some additional rules available on newer apple CPUs. rdar://41235346 llvm-svn: 342590
OpenPOWER on IntegriCloud