summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Target/AArch64
Commit message (Collapse)AuthorAgeFilesLines
...
* Revert r372893 "[CodeGen] Replace -max-jump-table-size with ↵Hans Wennborg2019-09-273-7/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | -max-jump-table-targets" This caused severe compile-time regressions, see PR43455. > Modern processors predict the targets of an indirect branch regardless of > the size of any jump table used to glean its target address. Moreover, > branch predictors typically use resources limited by the number of actual > targets that occur at run time. > > This patch changes the semantics of the option `-max-jump-table-size` to limit > the number of different targets instead of the number of entries in a jump > table. Thus, it is now renamed to `-max-jump-table-targets`. > > Before, when `-max-jump-table-size` was specified, it could happen that > cluster jump tables could have targets used repeatedly, but each one was > counted and typically resulted in tables with the same number of entries. > With this patch, when specifying `-max-jump-table-targets`, tables may have > different lengths, since the number of unique targets is counted towards the > limit, but the number of unique targets in tables is the same, but for the > last one containing the balance of targets. > > Differential revision: https://reviews.llvm.org/D60295 llvm-svn: 373060
* Speculative fix for gcc build.Peter Collingbourne2019-09-271-2/+4
| | | | llvm-svn: 373038
* hwasan: Compatibility fixes for short granules.Peter Collingbourne2019-09-272-75/+95
| | | | | | | | | | | | | | | | | | | | | | | | | We can't use short granules with stack instrumentation when targeting older API levels because the rest of the system won't understand the short granule tags stored in shadow memory. Moreover, we need to be able to let old binaries (which won't understand short granule tags) run on a new system that supports short granule tags. Such binaries will call the __hwasan_tag_mismatch function when their outlined checks fail. We can compensate for the binary's lack of support for short granules by implementing the short granule part of the check in the __hwasan_tag_mismatch function. Unfortunately we can't do anything about inline checks, but I don't believe that we can generate these by default on aarch64, nor did we do so when the ABI was fixed. A new function, __hwasan_tag_mismatch_v2, is introduced that lets code targeting the new runtime avoid redoing the short granule check. Because tag mismatches are rare this isn't important from a performance perspective; the main benefit is that it introduces a symbol dependency that prevents binaries targeting the new runtime from running on older (i.e. incompatible) runtimes. Differential Revision: https://reviews.llvm.org/D68059 llvm-svn: 373035
* Remove the AliasAnalysis argument in function areMemAccessesTriviallyDisjointChangpeng Fang2019-09-262-3/+2
| | | | | | | | | | Reviewers: arsenm Differential Revision: https://reviews.llvm.org/D58360 llvm-svn: 373024
* [AArch64][GlobalISel] Choose CCAssignFns per-argument for tail call loweringJessica Paquette2019-09-251-17/+33
| | | | | | | | | | | | | | | | | | | | | | | | | | When checking for tail call eligibility, we should use the correct CCAssignFn for each argument, rather than just checking if the caller/callee is varargs or not. This is important for tail call lowering with varargs. If we don't check it, then basically any varargs callee with parameters cannot be tail called on Darwin, for one thing. If the parameters are all guaranteed to be in registers, this should be entirely safe. On top of that, not checking for this could potentially make it so that we have the wrong stack offsets when checking for tail call eligibility. Also refactor some of the stuff for CCAssignFnForCall and pull it out into a helper function. Update call-translator-tail-call.ll to show that we can now correctly tail call on Darwin. Also add two extra tail call checks. The first verifies that we still respect the caller's stack size, and the second verifies that we still don't tail call when a varargs function has a memory argument. Differential Revision: https://reviews.llvm.org/D67939 llvm-svn: 372897
* [CodeGen] Replace -max-jump-table-size with -max-jump-table-targetsEvandro Menezes2019-09-253-8/+7
| | | | | | | | | | | | | | | | | | | | | | | Modern processors predict the targets of an indirect branch regardless of the size of any jump table used to glean its target address. Moreover, branch predictors typically use resources limited by the number of actual targets that occur at run time. This patch changes the semantics of the option `-max-jump-table-size` to limit the number of different targets instead of the number of entries in a jump table. Thus, it is now renamed to `-max-jump-table-targets`. Before, when `-max-jump-table-size` was specified, it could happen that cluster jump tables could have targets used repeatedly, but each one was counted and typically resulted in tables with the same number of entries. With this patch, when specifying `-max-jump-table-targets`, tables may have different lengths, since the number of unique targets is counted towards the limit, but the number of unique targets in tables is the same, but for the last one containing the balance of targets. Differential revision: https://reviews.llvm.org/D60295 llvm-svn: 372893
* [AArch64] Convert neon_ushl and neon_sshl with positive constants to VSHL.Florian Hahn2019-09-251-19/+4
| | | | | | | | | | | | | | | | | | I think we should be able to use shl instead of sshl and ushl for positive constant shift values, unless I am missing something. We already have the machinery in place to ensure we only replace nodes, if the shift value is positive and <= the element width. This is a generalization of an earlier patch rL372565. Reviewers: t.p.northover, samparker, dmgreen, anemet Reviewed By: anemet Differential Revision: https://reviews.llvm.org/D67955 llvm-svn: 372824
* [AArch64][GlobalISel] Tweak legalization rule for G_BSWAP to handle widening ↵Amara Emerson2019-09-251-1/+1
| | | | | | s16. llvm-svn: 372812
* Regex: Make "match" and "sub" const member functionsThomas Preud'homme2019-09-241-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The Regex "match" and "sub" member functions were previously not "const" because they wrote to the "error" member variable. This commit removes those assignments, and instead assumes that the validity of the regex is already known after the initial compilation of the regular expression. As a result, these member functions were possible to make "const". This makes it easier to do things like pre-compile Regexes up-front, and makes "match" and "sub" thread-safe. The error status is now returned as an optional output, which also makes the API of "match" and "sub" more consistent with each other. Also, some uses of Regex that could be refactored to be const were made const. Patch by Nicolas Guillemot Reviewers: jankratochvil, thopre Subscribers: llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D67241 llvm-svn: 372764
* MCRegisterInfo: Merge getLLVMRegNum and getLLVMRegNumFromEHPavel Labath2019-09-241-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The functions different in two ways: - getLLVMRegNum could return both "eh" and "other" dwarf register numbers, while getLLVMRegNumFromEH only returned the "eh" number. - getLLVMRegNum asserted if the register was not found, while the second function returned -1. The second distinction was pretty important, but it was very hard to infer that from the function name. Aditionally, for the use case of dumping dwarf expressions, we needed a function which can work with both kinds of number, but does not assert. This patch solves both of these issues by merging the two functions into one, returning an Optional<unsigned> value. While the same thing could be achieved by adding an "IsEH" argument to the (renamed) getLLVMRegNumFromEH function, it seemed better to avoid the confusion of two functions and put the choice of asserting into the hands of the caller -- if he checks the Optional value, he can safely process "untrusted" input, and if he blindly dereferences the Optional, he gets the assertion. I've updated all call sites to the new API, choosing between the two options according to the function they were calling originally, except that I've updated the usage in DWARFExpression.cpp to use the "safe" method instead, and added a test case which would have previously triggered an assertion failure when processing (incorrect?) dwarf expressions. Reviewers: dsanders, arsenm, JDevlieghere Subscribers: wdng, aprantl, javed.absar, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D67154 llvm-svn: 372710
* Cosmetic; don't use the magic constant 35 when HASH is more readable. This ↵Mark Murray2019-09-231-13/+13
| | | | | | | | | | | | | | | | matches other MCK__<THING>_* usage better. Summary: No functional change. This fixes a magic constant in MCK__*_... macros only. Reviewers: ostannard Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D67840 llvm-svn: 372599
* [Alignment] Get DataLayout::StackAlignment as AlignGuillaume Chatelet2019-09-231-3/+5
| | | | | | | | | | | | | | | | | | | | Summary: Internally it is needed to know if StackAlignment is set but we can expose it as llvm::Align. This is patch is part of a series to introduce an Alignment type. See this thread for context: http://lists.llvm.org/pipermail/llvm-dev/2019-July/133851.html See this patch for the introduction of the type: https://reviews.llvm.org/D64790 Reviewers: courbet Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D67852 llvm-svn: 372585
* [AArch64] support neon_sshl and neon_ushl in performIntrinsicCombine.Florian Hahn2019-09-231-0/+25
| | | | | | | | | | | | | | | | Try to generate ushll/sshll for aarch64_neon_ushl/aarch64_neon_sshl, if their first operand is extended and the second operand is a constant Also adds a few tests marked with FIXME, where we can further increase codegen. Reviewers: t.p.northover, samparker, dmgreen, anemet Reviewed By: anemet Differential Revision: https://reviews.llvm.org/D62308 llvm-svn: 372565
* [AArch64] AArch64StackTagging - Silence static analyzer dyn_cast<> null ↵Simon Pilgrim2019-09-221-1/+1
| | | | | | | | dereference warning. NFCI. The static analyzer is warning about potential null dereferences, but we should be able to use cast<> directly and if not assert will fire for us. llvm-svn: 372507
* [AArch64][GlobalISel] Implement selection for G_SHL of <2 x i64>Amara Emerson2019-09-211-1/+3
| | | | | | Simple continuation of existing selection support. llvm-svn: 372467
* [AArch64][GlobalISel] Selection support for G_ASHR of <2 x s64>Amara Emerson2019-09-211-1/+5
| | | | | | Just add an extra case to the existing selection logic. llvm-svn: 372466
* [AArch64][GlobalISel] Make <4 x s32> G_ASHR and G_LSHR legal.Amara Emerson2019-09-211-2/+6
| | | | llvm-svn: 372465
* [MTE] Handle MTE instructions in AArch64LoadStoreOptimizer.Evgeniy Stepanov2019-09-201-32/+98
| | | | | | | | | | | | | | Summary: Generate pre- and post-indexed forms of ST*G and STGP when possible. Reviewers: ostannard, vitalybuka Subscribers: kristof.beyls, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D67741 llvm-svn: 372412
* [aarch64] add def-pats for dot productSebastian Pop2019-09-201-0/+109
| | | | | | | | | This patch adds the patterns to select the dot product instructions. Tested on aarch64-linux with make check-all. Differential Revision: https://reviews.llvm.org/D67645 llvm-svn: 372408
* [AArch64] Fix formatting (NFC)Evandro Menezes2019-09-191-0/+1
| | | | llvm-svn: 372357
* Reapply r372285 "GlobalISel: Don't materialize immarg arguments to intrinsics"Matt Arsenault2019-09-192-3/+3
| | | | | | | | | This reverts r372314, reapplying r372285 and the commits which depend on it (r372286-r372293, and r372296-r372297) This was missing one switch to getTargetConstant in an untested case. llvm-svn: 372338
* [DAG] Add SelectionDAG::MaxRecursionDepth constantSimon Pilgrim2019-09-191-1/+1
| | | | | | | | | | As commented on D67557 we have a lot of uses of depth checks all using magic numbers. This patch adds the SelectionDAG::MaxRecursionDepth constant and moves over some general cases to use this explicitly. Differential Revision: https://reviews.llvm.org/D67711 llvm-svn: 372315
* Revert r372285 "GlobalISel: Don't materialize immarg arguments to intrinsics"Hans Wennborg2019-09-192-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This broke the Chromium build, causing it to fail with e.g. fatal error: error in backend: Cannot select: t362: v4i32 = X86ISD::VSHLI t392, Constant:i8<15> See llvm-commits thread of r372285 for details. This also reverts r372286, r372287, r372288, r372289, r372290, r372291, r372292, r372293, r372296, and r372297, which seemed to depend on the main commit. > Encode them directly as an imm argument to G_INTRINSIC*. > > Since now intrinsics can now define what parameters are required to be > immediates, avoid using registers for them. Intrinsics could > potentially want a constant that isn't a legal register type. Also, > since G_CONSTANT is subject to CSE and legalization, transforms could > potentially obscure the value (and create extra work for the > selector). The register bank of a G_CONSTANT is also meaningful, so > this could throw off future folding and legalization logic for AMDGPU. > > This will be much more convenient to work with than needing to call > getConstantVRegVal and checking if it may have failed for every > constant intrinsic parameter. AMDGPU has quite a lot of intrinsics wth > immarg operands, many of which need inspection during lowering. Having > to find the value in a register is going to add a lot of boilerplate > and waste compile time. > > SelectionDAG has always provided TargetConstant for constants which > should not be legalized or materialized in a register. The distinction > between Constant and TargetConstant was somewhat fuzzy, and there was > no automatic way to force usage of TargetConstant for certain > intrinsic parameters. They were both ultimately ConstantSDNode, and it > was inconsistently used. It was quite easy to mis-select an > instruction requiring an immediate. For SelectionDAG, start emitting > TargetConstant for these arguments, and using timm to match them. > > Most of the work here is to cleanup target handling of constants. Some > targets process intrinsics through intermediate custom nodes, which > need to preserve TargetConstant usage to match the intrinsic > expectation. Pattern inputs now need to distinguish whether a constant > is merely compatible with an operand or whether it is mandatory. > > The GlobalISelEmitter needs to treat timm as a special case of a leaf > node, simlar to MachineBasicBlock operands. This should also enable > handling of patterns for some G_* instructions with immediates, like > G_FENCE or G_EXTRACT. > > This does include a workaround for a crash in GlobalISelEmitter when > ARM tries to uses "imm" in an output with a "timm" pattern source. llvm-svn: 372314
* GlobalISel: Don't materialize immarg arguments to intrinsicsMatt Arsenault2019-09-192-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Encode them directly as an imm argument to G_INTRINSIC*. Since now intrinsics can now define what parameters are required to be immediates, avoid using registers for them. Intrinsics could potentially want a constant that isn't a legal register type. Also, since G_CONSTANT is subject to CSE and legalization, transforms could potentially obscure the value (and create extra work for the selector). The register bank of a G_CONSTANT is also meaningful, so this could throw off future folding and legalization logic for AMDGPU. This will be much more convenient to work with than needing to call getConstantVRegVal and checking if it may have failed for every constant intrinsic parameter. AMDGPU has quite a lot of intrinsics wth immarg operands, many of which need inspection during lowering. Having to find the value in a register is going to add a lot of boilerplate and waste compile time. SelectionDAG has always provided TargetConstant for constants which should not be legalized or materialized in a register. The distinction between Constant and TargetConstant was somewhat fuzzy, and there was no automatic way to force usage of TargetConstant for certain intrinsic parameters. They were both ultimately ConstantSDNode, and it was inconsistently used. It was quite easy to mis-select an instruction requiring an immediate. For SelectionDAG, start emitting TargetConstant for these arguments, and using timm to match them. Most of the work here is to cleanup target handling of constants. Some targets process intrinsics through intermediate custom nodes, which need to preserve TargetConstant usage to match the intrinsic expectation. Pattern inputs now need to distinguish whether a constant is merely compatible with an operand or whether it is mandatory. The GlobalISelEmitter needs to treat timm as a special case of a leaf node, simlar to MachineBasicBlock operands. This should also enable handling of patterns for some G_* instructions with immediates, like G_FENCE or G_EXTRACT. This does include a workaround for a crash in GlobalISelEmitter when ARM tries to uses "imm" in an output with a "timm" pattern source. llvm-svn: 372285
* [AArch64][GlobalISel] Support lowering musttail callsJessica Paquette2019-09-181-10/+24
| | | | | | | | | | | | | | | | | | | | | Since we now lower most tail calls, it makes sense to support musttail. Instead of always falling back to SelectionDAG, only fall back when a musttail call was not able to be emitted as a tail call. Once we can handle most incoming and outgoing arguments, we can change this to a `report_fatal_error` like in ISelLowering. Remove the assert that we don't have varargs and a musttail, and replace it with a return false. Implementing this requires that we implement `saveVarArgRegisters` from AArch64ISelLowering, which is an entirely different patch. Add GlobalISel lines to vararg-tallcall.ll to make sure that we produce correct code. Right now we only fall back, but eventually this will be relevant. Differential Revision: https://reviews.llvm.org/D67681 llvm-svn: 372273
* [AArch64] Don't implicitly enable global isel on Darwin if code-model==large.Lang Hames2019-09-181-2/+4
| | | | | | | | | | | | | | | | Summary: AArch64 GlobalISel doesn't support MachO's large code model, so this patch adds a check for that combination before implicitly enabling it. Reviewers: paquette Subscribers: kristof.beyls, ributzka, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D67724 llvm-svn: 372256
* Revert "[AArch64][DebugInfo] Do not recompute CalleeSavedStackSize"Krasimir Georgiev2019-09-181-4/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This reverts commit r372204. This change causes build bot failures under msan: http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/35236/steps/check-llvm%20msan/logs/stdio: ``` FAIL: LLVM :: DebugInfo/AArch64/asan-stack-vars.mir (19531 of 33579) ******************** TEST 'LLVM :: DebugInfo/AArch64/asan-stack-vars.mir' FAILED ******************** Script: -- : 'RUN: at line 1'; /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/llc -O0 -start-before=livedebugvalues -filetype=obj -o - /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/DebugInfo/AArch64/asan-stack-vars.mir | /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/llvm-dwarfdump -v - | /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/FileCheck /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/DebugInfo/AArch64/asan-stack-vars.mir -- Exit Code: 2 Command Output (stderr): -- ==62894==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0xdfcafb in llvm::AArch64FrameLowering::resolveFrameOffsetReference(llvm::MachineFunction const&, int, bool, unsigned int&, bool, bool) const /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/Target/AArch64/AArch64FrameLowering.cpp:1658:3 #1 0xdfae8a in resolveFrameIndexReference /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/Target/AArch64/AArch64FrameLowering.cpp:1580:10 #2 0xdfae8a in llvm::AArch64FrameLowering::getFrameIndexReference(llvm::MachineFunction const&, int, unsigned int&) const /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/Target/AArch64/AArch64FrameLowering.cpp:1536 #3 0x46642c1 in (anonymous namespace)::LiveDebugValues::extractSpillBaseRegAndOffset(llvm::MachineInstr const&) /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/CodeGen/LiveDebugValues.cpp:582:21 #4 0x4647cb3 in transferSpillOrRestoreInst /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/CodeGen/LiveDebugValues.cpp:883:11 #5 0x4647cb3 in process /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/CodeGen/LiveDebugValues.cpp:1079 #6 0x4647cb3 in (anonymous namespace)::LiveDebugValues::ExtendRanges(llvm::MachineFunction&) /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/CodeGen/LiveDebugValues.cpp:1361 #7 0x463ac0e in (anonymous namespace)::LiveDebugValues::runOnMachineFunction(llvm::MachineFunction&) /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/CodeGen/LiveDebugValues.cpp:1415:18 #8 0x4854ef0 in llvm::MachineFunctionPass::runOnFunction(llvm::Function&) /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/CodeGen/MachineFunctionPass.cpp:73:13 #9 0x53b0b01 in llvm::FPPassManager::runOnFunction(llvm::Function&) /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1648:27 #10 0x53b15f6 in llvm::FPPassManager::runOnModule(llvm::Module&) /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1685:16 #11 0x53b298d in runOnModule /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1750:27 #12 0x53b298d in llvm::legacy::PassManagerImpl::run(llvm::Module&) /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1863 #13 0x905f21 in compileModule(char**, llvm::LLVMContext&) /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/tools/llc/llc.cpp:601:8 #14 0x8fdc4e in main /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/tools/llc/llc.cpp:355:22 #15 0x7f67673632e0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202e0) #16 0x882369 in _start (/b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/llc+0x882369) MemorySanitizer: use-of-uninitialized-value /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/Target/AArch64/AArch64FrameLowering.cpp:1658:3 in llvm::AArch64FrameLowering::resolveFrameOffsetReference(llvm::MachineFunction const&, int, bool, unsigned int&, bool, bool) const Exiting error: -: The file was not recognized as a valid object file FileCheck error: '-' is empty. FileCheck command line: /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/FileCheck /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/DebugInfo/AArch64/asan-stack-vars.mir ``` Reviewers: bkramer Reviewed By: bkramer Subscribers: sdardis, aprantl, kristof.beyls, jrtc27, atanasyan, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D67710 llvm-svn: 372228
* [AArch64][DebugInfo] Do not recompute CalleeSavedStackSizeSander de Smalen2019-09-181-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch fixes a bug exposed by D65653 where a subsequent invocation of `determineCalleeSaves` ends up with a different size for the callee save area, leading to different frame-offsets in debug information. In the invocation by PEI, `determineCalleeSaves` tries to determine whether it needs to spill an extra callee-saved register to get an emergency spill slot. To do this, it calls 'estimateStackSize' and manually adds the size of the callee-saves to this. PEI then allocates the spill objects for the callee saves and the remaining frame layout is calculated accordingly. A second invocation in LiveDebugValues causes estimateStackSize to return the size of the stack frame including the callee-saves. Given that the size of the callee-saves is added to this, these callee-saves are counted twice, which leads `determineCalleeSaves` to believe the stack has become big enough to require spilling an extra callee-save as emergency spillslot. It then updates CalleeSavedStackSize with a larger value. Since CalleeSavedStackSize is used in the calculation of the frame offset in getFrameIndexReference, this leads to incorrect offsets for variables/locals when this information is recalculated after PEI. Reviewers: omjavaid, eli.friedman, thegameg, efriedma Reviewed By: efriedma Differential Revision: https://reviews.llvm.org/D66935 llvm-svn: 372204
* [AArch64][GlobalISel] Support -tailcalloptJessica Paquette2019-09-171-23/+105
| | | | | | | | | | | | | | | | | | | | This adds support for `-tailcallopt` tail calls to CallLowering. This piggy-backs off the changes from D67577, since doing it without a bit of refactoring gets extremely ugly. Support is basically ported from AArch64ISelLowering. The main difference here is that tail calls in `-tailcallopt` change the ABI, so there's some extra bookkeeping for the stack. Show that we are correctly lowering these by updating tail-call.ll. Also show that we don't do anything strange in general by updating fastcc-reserved.ll, which passes `-tailcallopt`, but doesn't emit any tail calls. Differential Revision: https://reviews.llvm.org/D67580 llvm-svn: 372177
* AArch64CallLowering::lowerCall(): fix build by not passing InArgs into ↵Roman Lebedev2019-09-171-1/+1
| | | | | | lowerTailCall() llvm-svn: 372172
* [AArch64][GlobalISel][NFC] Refactor tail call lowering codeJessica Paquette2019-09-172-31/+74
| | | | | | | | | | When you begin implementing -tailcallopt, this gets somewhat hairy. Refactor the call lowering code so that the tail call lowering stuff gets its own function. Differential Revision: https://reviews.llvm.org/D67577 llvm-svn: 372164
* Hide implementation details in namespaces.Benjamin Kramer2019-09-171-1/+1
| | | | llvm-svn: 372113
* [SVE][MVT] Fixed-length vector MVT rangesGraham Hunter2019-09-171-4/+4
| | | | | | | | | | | | | | | | | * Reordered MVT simple types to group scalable vector types together. * New range functions in MachineValueType.h to only iterate over the fixed-length int/fp vector types. * Stopped backends which don't support scalable vector types from iterating over scalable types. Reviewers: sdesmalen, greened Reviewed By: greened Differential Revision: https://reviews.llvm.org/D66339 llvm-svn: 372099
* [SVE][Inline-Asm] Add constraints for SVE predicate registersKerry McLaughlin2019-09-163-1/+43
| | | | | | | | | | | | | | | | | | | Summary: Adds the following inline asm constraints for SVE: - Upl: One of the low eight SVE predicate registers, P0 to P7 inclusive - Upa: SVE predicate register with full range, P0 to P15 Reviewers: t.p.northover, sdesmalen, rovka, momchil.velikov, cameron.mcinally, greened, rengolin Reviewed By: rovka Subscribers: javed.absar, tschuett, rkruppe, psnobl, cfe-commits, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D66524 llvm-svn: 371967
* [AArch64] Some more FP16 FMA pattern matchingSjoerd Meijer2019-09-161-2/+19
| | | | | | | | | After our previous machinecombiner exercises (rL371321, rL371818, rL371833), we were still missing a few FP16 FMA patterns. Differential Revision: https://reviews.llvm.org/D67576 llvm-svn: 371960
* [AArch64][GlobalISel] Tail call memory intrinsicsJessica Paquette2019-09-131-1/+3
| | | | | | | | | | | | | | | | | | | | | | Because memory intrinsics are handled differently than other calls, we need to check them for tail call eligiblity in the legalizer. This allows us to still inline them when it's beneficial to do so, but also tail call when possible. This adds simple tail calling support for when the intrinsic is followed by a return. It ports the attribute checks from `TargetLowering::isInTailCallPosition` into a similarly-named function in LegalizerHelper.cpp. The target-specific `isUsedByReturnOnly` hook is not ported here. Update tailcall-mem-intrinsics.ll to show that GlobalISel can now tail call memory intrinsics. Update legalize-memcpy-et-al.mir to have a case where we don't tail call. Differential Revision: https://reviews.llvm.org/D67566 llvm-svn: 371893
* [aarch64] move custom isel of extract_vector_elt to td file - NFCSebastian Pop2019-09-132-43/+10
| | | | | | | | | | | | | | | | | | | In preparation for def-pat selection of dot product instructions, this patch moves the custom instruction selection of extract_vector_elt to the td file. Without this change it is impossible to catch a pattern that starts with an extract_vector_elt: the custom cpp code is executed first ahead of the patterns in the td files that are only executed at the end of the switch statement in SelectCode(Node). With this patch applied, it becomes possible to select a different pattern that starts with extract_vector_elt by selecting a higher complexity than this pattern. The patch has been tested on aarch64-linux with make check-all. Differential Revision: https://reviews.llvm.org/D67497 llvm-svn: 371887
* AArch64: fix EXPENSIVE_CHECKS for arm64_32.Tim Northover2019-09-131-1/+1
| | | | | | | For some reason I'd decided to mark the end-result of a GOT load as dead. It's clearly not (necessarily). llvm-svn: 371883
* [AArch64][GlobalISel] Add support for sibcalling callees with varargsJessica Paquette2019-09-131-6/+19
| | | | | | | | | | | | | | | This adds support for tail calling callees with varargs, equivalent to how it is done in AArch64ISelLowering. This only works for sibling calls, and does not add the necessary support for musttail with varargs. (See r345641 for equivalent ISelLowering support.) This should be implemented when we stop falling back on musttail. Update call-translator-tail-call.ll to show that we can now tail call varargs. Differential Revision: https://reviews.llvm.org/D67518 llvm-svn: 371868
* [AArch64] MachineCombiner FMA matching. NFC.Sjoerd Meijer2019-09-131-325/+112
| | | | | | | | | Follow-up of rL371321 that added some more FP16 FMA patterns, and an attempt to reduce the copy-pasting and make this more readable. Differential Revision: https://reviews.llvm.org/D67403 llvm-svn: 371818
* [AArch64][GlobalISel] Support tail calling with swiftself parametersJessica Paquette2019-09-121-5/+32
| | | | | | | | | | | | | | | | Swiftself uses a callee-saved register. We can tail call when the register used in the caller and callee is the same. This behaviour is equivalent to that in `TargetLowering::parametersInCSRMatch`. Update call-translator-tail-call.ll to verify that we can do this. When we support inline assembly, we can write a check similar to the one in the general swiftself.ll. For now, we need to verify that we get the correct COPY instruction after call lowering. Differential Revision: https://reviews.llvm.org/D67511 llvm-svn: 371788
* [AArch64][GlobalISel] Support sibling calls with outgoing argumentsJessica Paquette2019-09-122-22/+102
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds support for lowering sibling calls with outgoing arguments. e.g ``` define void @foo(i32 %a) ``` Support is ported from AArch64ISelLowering's `isEligibleForTailCallOptimization`. The only thing that is missing is a full port of `TargetLowering::parametersInCSRMatch`. So, if we're using swiftself, we'll never tail call. - Rename `analyzeCallResult` to `analyzeArgInfo`, since the function is now used for both outgoing and incoming arguments - Teach `OutgoingArgHandler` about tail calls. Tail calls use frame indices for stack arguments. - Teach `lowerFormalArguments` to set the bytes in the caller's stack argument area. This is used later to check if the tail call's parameters will fit on the caller's stack. - Add `areCalleeOutgoingArgsTailCallable` to perform the eligibility check on the callee's outgoing arguments. For testing: - Update call-translator-tail-call to verify that we can now tail call with outgoing arguments, use G_FRAME_INDEX for stack arguments, and respect the size of the caller's stack - Remove GISel-specific check lines from speculation-hardening.ll, since GISel now tail calls like the other selectors - Add a GISel test line to tailcall-string-rvo.ll since we can tail call in that test now - Add a GISel test line to tailcall_misched_graph.ll since we tail call there now. Add specific check lines for GISel, since the debug output from the machine-scheduler differs with GlobalISel. The dependency still holds, but the output comes out in a different order. Differential Revision: https://reviews.llvm.org/D67471 llvm-svn: 371780
* AArch64: support arm64_32, an ILP32 slice for watchOS.Tim Northover2019-09-1217-74/+354
| | | | | | | | This is the main CodeGen patch to support the arm64_32 watchOS ABI in LLVM. FastISel is mostly disabled for now since it would generate incorrect code for ILP32. llvm-svn: 371722
* [GlobalISel][AArch64] Check caller for swifterror params in tailcall eligibilityJessica Paquette2019-09-111-3/+7
| | | | | | | | | | | | | | | | Before, we only checked the callee for swifterror. However, we should also be checking the caller to see if it has a swifterror parameter. Since we don't currently handle outgoing arguments, this didn't show up in the swifterror.ll testcase. Also, remove the swifterror checks from call-translator-tail-call.ll, since they are covered by the existing swifterror testing. Better to have it all in one place. Differential Revision: https://reviews.llvm.org/D67465 llvm-svn: 371692
* [Alignment][NFC] use llvm::Align for AsmPrinter::EmitAlignmentGuillaume Chatelet2019-09-111-1/+1
| | | | | | | | | | | | | | | | | Summary: This is patch is part of a series to introduce an Alignment type. See this thread for context: http://lists.llvm.org/pipermail/llvm-dev/2019-July/133851.html See this patch for the introduction of the type: https://reviews.llvm.org/D64790 Reviewers: courbet Subscribers: dschuff, sdardis, nemanjai, hiraditya, kbarton, jrtc27, MaskRay, atanasyan, jsji, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D67443 llvm-svn: 371616
* [GlobalISel] When a tail call is emitted in a block, stop translating itJessica Paquette2019-09-101-11/+0
| | | | | | | | | | | | | | | | | | | | | | | | This fixes a crash in tail call translation caused by assume and lifetime_end intrinsics. It's possible to have instructions other than a return after a tail call which will still have `Analysis::isInTailCallPosition` return true. (Namely, lifetime_end and assume intrinsics.) If we emit a tail call, we should stop translating instructions in the block. Otherwise, we can end up emitting an extra return, or dead instructions in general. This makes the verifier unhappy, and is generally unfortunate for codegen. This also removes the code from AArch64CallLowering that checks if we have a tail call when lowering a return. This is covered by the new code now. Also update call-translator-tail-call.ll to show that we now properly tail call in the presence of lifetime_end and assume. Differential Revision: https://reviews.llvm.org/D67415 llvm-svn: 371572
* [AArch64][GlobalISel] Support sibling calls with mismatched calling conventionsJessica Paquette2019-09-102-17/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | Add support for sibcalling calls whose calling convention differs from the caller's. - Port over `CCState::resultsCombatible` from CallingConvLower.cpp into CallLowering. This is used to verify that the way the caller and callee CC handle incoming arguments matches up. - Add `CallLowering::analyzeCallResult`. This is basically a port of `CCState::AnalyzeCallResult`, but using `ArgInfo` rather than `ISD::InputArg`. - Add `AArch64CallLowering::doCallerAndCalleePassArgsTheSameWay`. This checks that the calling conventions are compatible, and that the caller and callee preserve the same registers. For testing: - Update call-translator-tail-call.ll to show that we can now handle this. - Add a GISel line to tailcall-ccmismatch.ll to show that we will not tail call when the regmasks don't line up. Differential Revision: https://reviews.llvm.org/D67361 llvm-svn: 371570
* [GlobalISel][AArch64] Handle tail calls with non-void return typesJessica Paquette2019-09-091-15/+9
| | | | | | | | | | | | | | | Just return once you emit the call, which is exactly what SelectionDAG does in this situation. Update call-translator-tail-call.ll. Also update dllimport.ll to show that we tail call here in GISel again. Add -verify-machineinstrs to the GISel line too, to defend against verifier failures. Differential revision: https://reviews.llvm.org/D67282 llvm-svn: 371425
* [AArch64][SVE] Implement abs and neg intrinsicsCullen Rhodes2019-09-092-3/+17
| | | | | | | | | | | | | | | | Summary: This patch implements two arithmetic intrinsics: * int_aarch64_sve_abs * int_aarch64_sve_neg testing the support for scalable vector types in intrinsics added in D65930. Reviewed By: greened Differential Revision: https://reviews.llvm.org/D65931 llvm-svn: 371388
* GlobalISel: add combiner to form indexed loads.Tim Northover2019-09-093-12/+30
| | | | | | | | | | | Loosely based on DAGCombiner version, but this part is slightly simpler in GlobalIsel because all address calculation is performed by G_GEP. That makes the inc/dec distinction moot so there's just pre/post to think about. No targets can handle it yet so testing is via a special flag that overrides target hooks. llvm-svn: 371384
OpenPOWER on IntegriCloud