summaryrefslogtreecommitdiffstats
path: root/llvm/lib
Commit message (Collapse)AuthorAgeFilesLines
...
* AMDGPU: Fix not using s33 for scratch wave offset in kernelsMatt Arsenault2019-06-211-7/+11
| | | | | | Fixes missing piece from r363990. llvm-svn: 364099
* [X86] Add DAG combine to turn (vzmovl (insert_subvector undef, X, 0)) into ↵Craig Topper2019-06-213-57/+16
| | | | | | | | | | | | | | | | (insert_subvector allzeros, (vzmovl X), 0) 128/256 bit scalar_to_vectors are canonicalized to (insert_subvector undef, (scalar_to_vector), 0). We have isel patterns that try to match this pattern being used by a vzmovl to use a 128-bit instruction and a subreg_to_reg. This patch detects the insert_subvector undef portion of this and pulls it through the vzmovl, creating a narrower vzmovl and an insert_subvector allzeroes. We can then match the insertsubvector into a subreg_to_reg operation by itself. Then we can fall back on existing (vzmovl (scalar_to_vector)) patterns. Note, while the scalar_to_vector case is the motivating case I didn't restrict to just that case. I'm also wondering about shrinking any 256/512 vzmovl to an extract_subvector+vzmovl+insert_subvector(allzeros) but I fear that would have bad implications to shuffle combining. I also think there is more canonicalization we can do with vzmovl with loads or scalar_to_vector with loads to create vzload. Differential Revision: https://reviews.llvm.org/D63512 llvm-svn: 364095
* [X86] Don't mark v64i8/v32i16 ISD::SELECT as custom unless they are legal types.Craig Topper2019-06-211-7/+4
| | | | | | | | | We don't have any Custom handling during type legalization. Only operation legalization. Fixes PR42355 llvm-svn: 364093
* [X86] Add a debug print of the node in the default case for unhandled ↵Craig Topper2019-06-211-0/+4
| | | | | | | | | | opcodes in ReplaceNodeResults. This should be unreachable, but bugs can make it reachable. This adds a debug print so we can see the bad node in the output when the llvm_unreachable triggers. llvm-svn: 364091
* [X86][AVX] Combine INSERT_SUBVECTOR(SRC0, EXTRACT_SUBVECTOR(SRC1)) as shuffleSimon Pilgrim2019-06-211-2/+15
| | | | | | Subvector shuffling often ends up as insert/extract subvector. llvm-svn: 364090
* [AArch64][GlobalISel] Implement selection support for the new G_JUMP_TABLE ↵Amara Emerson2019-06-212-0/+52
| | | | | | | | | | and G_BRJT ops. With this we can now fully code generate jump tables, which is important for code size. Differential Revision: https://reviews.llvm.org/D63223 llvm-svn: 364086
* [GlobalISel][IRTranslator] Change switch table translation to generate jump ↵Amara Emerson2019-06-212-54/+441
| | | | | | | | | | | | | | | | | | | | | | | | | | tables and range checks. This change makes use of the newly refactored SwitchLoweringUtils code from SelectionDAG to in order to generate jump tables and range checks where appropriate. Much of this code is ported from SDAG with some modifications. We generate G_JUMP_TABLE and G_BRJT instructions when JT opportunities are found. This means that targets which previously relied on the naive one MBB per case stmt translation will now start falling back until they add support for the new opcodes. For range checks, we don't generate any previously unused operations. This just recognizes contiguous ranges of case values and generates a single block per range. Single case value blocks are just a special case of ranges so we get that support almost for free. There are still some optimizations missing that I haven't ported over, and bit-tests are also unimplemented. This patch series is already complex enough. Actual arm64 support for selection of jump tables is coming in a later patch. Differential Revision: https://reviews.llvm.org/D63169 llvm-svn: 364085
* [SLP] Look-ahead operand reordering heuristic.Simon Pilgrim2019-06-211-46/+232
| | | | | | | | | | This patch introduces a new heuristic for guiding operand reordering. The new "look-ahead" heuristic can look beyond the immediate predecessors. This helps break ties when the immediate predecessors have identical opcodes (see lit test for an example). Committed on behalf of @vporpo (Vasileios Porpodas) Differential Revision: https://reviews.llvm.org/D60897 llvm-svn: 364084
* [X86] Use vmovq for v4i64/v4f64/v8i64/v8f64 vzmovl.Craig Topper2019-06-212-53/+35
| | | | | | | | | | | | | | We already use vmovq for v2i64/v2f64 vzmovl. But we were using a blendpd+xorpd for v4i64/v4f64/v8i64/v8f64 under opt speed. Or movsd+xorpd under optsize. I think the blend with 0 or movss/d is only needed for vXi32 where we don't have an instruction that can move 32 bits from one xmm to another while zeroing upper bits. movq is no worse than blendpd on any known CPUs. llvm-svn: 364079
* [DAGCombine] narrowExtractedVectorBinOp - pull out repeated getOpcode(). NFCI.Simon Pilgrim2019-06-211-2/+2
| | | | llvm-svn: 364076
* [AArch64][GlobalISel] Make s8 and s16 G_CONSTANTs legal.Amara Emerson2019-06-212-4/+7
| | | | | | | | | | | | | | | | | | | | | We sometimes get poor code size because constants of types < 32b are legalized as 32 bit G_CONSTANTs with a truncate to fit. This works but means that the localizer can no longer sink them (although it's possible to extend it to do so). On AArch64 however s8 and s16 constants can be selected in the same way as s32 constants, with a mov pseudo into a W register. If we make s8 and s16 constants legal then we can avoid unnecessary truncates, they can be CSE'd, and the localizer can sink them as normal. There is a caveat: if the user of a smaller constant has to widen the sources, we end up with an anyext of the smaller typed G_CONSTANT. This can cause regressions because of the additional extend and missed pattern matching. To remedy this, there's a new artifact combiner to generate the wider G_CONSTANT if it's legal for the target. Differential Revision: https://reviews.llvm.org/D63587 llvm-svn: 364075
* [AMDGPU] hazard recognizer for fp atomic to s_denorm_modeStanislav Mekhanoshin2019-06-219-28/+112
| | | | | | | | | This requires 3 wait states unless there is a wait or VALU in between. Differential Revision: https://reviews.llvm.org/D63619 llvm-svn: 364074
* [InstCombine] (1 << (C - x)) -> ((1 << C) >> x) if C is bitwidth - 1David Bolvansky2019-06-211-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: ``` %a = sub i32 31, %x %r = shl i32 1, %a => %d = shl i32 1, 31 %r = lshr i32 %d, %x Done: 1 Optimization is correct! ``` https://rise4fun.com/Alive/btZm Reviewers: spatel, lebedev.ri, nikic Reviewed By: lebedev.ri Subscribers: llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D63652 llvm-svn: 364073
* [X86] isBinOp - move commutative ops to isCommutativeBinOp. NFCI.Simon Pilgrim2019-06-211-6/+6
| | | | | | TargetLoweringBase::isBinOp checks isCommutativeBinOp as a fallback, so don't duplicate. llvm-svn: 364072
* Fix MSVC "result of 32-bit shift implicitly converted to 64 bits" warning. NFCI.Simon Pilgrim2019-06-211-1/+1
| | | | llvm-svn: 364068
* [InstCombine] cttz(abs(x)) -> cttz(x)David Bolvansky2019-06-211-4/+15
| | | | | | | | | | | | Summary: Signedness does not change number of trailing zeros. Reviewers: spatel, lebedev.ri, nikic Reviewed By: lebedev.ri Differential Revision: https://reviews.llvm.org/D63546 llvm-svn: 364064
* [GVNSink] prevent crashing on mismatched instructions (PR42346)Sanjay Patel2019-06-211-0/+9
| | | | | | | Patch based on suggestion by James Molloy (@jmolloy) in: https://bugs.llvm.org/show_bug.cgi?id=42346 llvm-svn: 364062
* [DAGCombine] narrowInsertExtractVectorBinOp - reuse "extract from insert" ↵Simon Pilgrim2019-06-211-11/+15
| | | | | | | | detection code. Move the "extract from insert detection code" into a lambda helper function. llvm-svn: 364059
* [Scalarizer] Propagate IR flagsJay Foad2019-06-211-4/+6
| | | | | | | | | | | | | | | | | | Summary: The motivation for this was to propagate fast-math flags like nnan and ninf on vector floating point operations to the corresponding scalar operations to take advantage of follow-on optimizations. But I think the same argument applies to all of our IR flags: if they apply to the vector operation then they also apply to all the individual scalar operations, and they might enable follow-on optimizations. Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D63593 llvm-svn: 364051
* [RISCV] Add RISCV-specific TargetTransformInfoSam Elliott2019-06-216-2/+154
| | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: LLVM Allows Targets to provide information that guides optimisations made to LLVM IR. This is done with callbacks on a TargetTransformInfo object. This patch adds a TargetTransformInfo class for RISC-V. This will allow us to implement RISC-V specific callbacks as they become necessary. This commit also adds the getIntImmCost callbacks, and tests them with a simple constant hoisting test. Our immediate costs are on the conservative side, for the moment, but we prevent hoisting in most circumstances anyway. Previous review was on D63007 Reviewers: asb, luismarques Reviewed By: asb Subscribers: ributzka, MaskRay, llvm-commits, Jim, benna, psnobl, jocewei, PkmX, rkruppe, the_o, brucehoult, MartinMosbeck, rogfer01, edward-jones, zzheng, jrtc27, shiva0217, kito-cheng, niosHD, sabuasal, apazos, simoncook, johnrusso, rbar, hiraditya, mgorny Tags: #llvm Differential Revision: https://reviews.llvm.org/D63433 llvm-svn: 364046
* [ARM] Add MVE 64-bit GPR <-> vector move instructions.Simon Tatham2019-06-215-0/+216
| | | | | | | | | | | | | | | | | | | | | | | | | | | | These instructions let you load half a vector register at once from two general-purpose registers, or vice versa. The assembly syntax for these instructions mentions the vector register name twice. For the move _into_ a vector register, the MC operand list also has to mention the register name twice (once as the output, and once as an input to represent where the unchanged half of the output register comes from). So we can conveniently assign one of the two asm operands to be the output $Qd, and the other $QdSrc, which avoids confusing the auto-generated AsmMatcher too much. For the move _from_ a vector register, there's no way to get round the fact that both instances of that register name have to be inputs, so we need a custom AsmMatchConverter to avoid generating two separate output MC operands. (And even that wouldn't have worked if it hadn't been for D60695.) Reviewers: dmgreen, samparker, SjoerdMeijer, t.p.northover Subscribers: javed.absar, kristof.beyls, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D62679 llvm-svn: 364041
* [ARM] Add MVE vector instructions that take a scalar input.Simon Tatham2019-06-216-2/+440
| | | | | | | | | | | | | | | | | | | This adds the `MVE_qDest_rSrc` superclass and all its instances, plus a few other instructions that also take a scalar input register or two. I've also belatedly added custom diagnostic messages to the operand classes for odd- and even-numbered GPRs, which required matching changes in two of the existing MVE assembly test files. Reviewers: dmgreen, samparker, SjoerdMeijer, t.p.northover Subscribers: javed.absar, kristof.beyls, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D62678 llvm-svn: 364040
* Fix a crash with assembler source and -g.Paul Robinson2019-06-211-2/+2
| | | | | | | | | | | | | | | llvm-mc or clang with -g normally produces debug info describing the assembler source itself; however, if that source already contains some .file/.loc directives, we should instead emit the debug info described by those directives. For certain assembler sources seen in the wild (particularly in the Chrome build) this was causing a crash due to incorrect assumptions about legal sequences of assembler source text. Fixes PR38994. Differential Revision: https://reviews.llvm.org/D63573 llvm-svn: 364039
* [X86] X86ISD::ANDNP is a (non-commutative) binopSimon Pilgrim2019-06-211-0/+2
| | | | | | The sat add/sub tests still have unnecessary extract_subvector((vandnps ymm, ymm), 0) uses that should be split to (vandnps (extract_subvector(ymm, 0), extract_subvector(ymm, 0)), but its getting better. llvm-svn: 364038
* [ARM] Add a batch of similarly encoded MVE instructions.Simon Tatham2019-06-213-1/+345
| | | | | | | | | | | | | | | | | | | | | | | Summary: This adds the `MVE_qDest_qSrc` superclass and all instructions that inherit from it. It's not the complete class of _everything_ with a q-register as both destination and source; it's a subset of them that all have similar encodings (but it would have been hopelessly unwieldy to call it anything like MVE_111x11100). This category includes add/sub with carry; long multiplies; halving multiplies; multiply and accumulate, and some more complex instructions. Reviewers: dmgreen, samparker, SjoerdMeijer, t.p.northover Subscribers: javed.absar, kristof.beyls, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D62677 llvm-svn: 364037
* [X86] createMMXBuildVector - call with BuildVectorSDNode directly. NFCI.Simon Pilgrim2019-06-211-7/+5
| | | | llvm-svn: 364030
* [ARM] Fix -Wimplicit-fallthrough after D62675Fangrui Song2019-06-211-0/+2
| | | | llvm-svn: 364028
* [ARM] Add MVE vector compare instructions.Simon Tatham2019-06-213-6/+201
| | | | | | | | | | | | | | | | | | Summary: These take a pair of vector register to compare, and a comparison type (written in the form of an Arm condition suffix); they output a vector of booleans in the VPR register, where predication can conveniently use them. Reviewers: dmgreen, samparker, SjoerdMeijer, t.p.northover Subscribers: javed.absar, kristof.beyls, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D62676 llvm-svn: 364027
* [X86] combineAndnp - use isNOT instead of manually checking for (XOR x, -1)Simon Pilgrim2019-06-211-5/+3
| | | | llvm-svn: 364026
* [Symbolize] Avoid lifetime extension and simplify std::map find/insert. NFCFangrui Song2019-06-211-33/+26
| | | | llvm-svn: 364025
* [X86] foldVectorXorShiftIntoCmp - use isConstOrConstSplat. NFCI.Simon Pilgrim2019-06-211-7/+4
| | | | | | Use the isConstOrConstSplat helper instead of inspecting the build vector manually. llvm-svn: 364024
* [X86][AVX] isNOT - handle concat_vectors(xor X, -1, xor Y, -1) patternSimon Pilgrim2019-06-211-0/+10
| | | | llvm-svn: 364022
* [ARM] Add a batch of MVE floating-point instructions.Simon Tatham2019-06-213-4/+456
| | | | | | | | | | | | | | | | | Summary: This includes floating-point basic arithmetic (add/sub/multiply), complex add/multiply, unary negation and absolute value, rounding to integer value, and conversion to/from integer formats. Reviewers: dmgreen, samparker, SjoerdMeijer, t.p.northover Subscribers: javed.absar, kristof.beyls, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D62675 llvm-svn: 364013
* Simplify std::lower_bound with llvm::{bsearch,lower_bound}. NFCFangrui Song2019-06-2124-90/+57
| | | | llvm-svn: 364006
* [MIPS GlobalISel] Fix -Wunused-variable in -DLLVM_ENABLE_ASSERTIONS=off ↵Fangrui Song2019-06-212-2/+4
| | | | | | builds after D63541 llvm-svn: 364003
* [GlobalISel][Localizer] Allow localization of G_INTTOPTR and chains of ↵Amara Emerson2019-06-211-14/+15
| | | | | | | | | | | | | | | | | | | | instructions. G_INTTOPTR can prevent the localizer from moving G_CONSTANTs, but since it's essentially a side effect free cast instruction we can remat both instructions. This patch changes the localizer to enable localization of the chains by iterating over the entry block instructions in reverse order. That way, uses will localized first, and then the defs are free to be localized as well. This also changes the previous SmallPtrSet of localized instructions to use a SetVector instead. We're dealing with pointers and need deterministic iteration order. Overall, this change improves ARM64 -O0 CTMark code size by around 0.7% geomean. Differential Revision: https://reviews.llvm.org/D63630 llvm-svn: 364001
* [Reassociate] Remove bogus assert reported in PR42349.Cameron McInally2019-06-201-5/+1
| | | | | | | | Also, add a FIXME for the unsafe transform on a unary FNeg. A unary FNeg can only be transformed to a FMul by -1.0 when the nnan flag is present. The unary FNeg project is a WIP, so the unsafe transformation is acceptable until that work is complete. The bogus assert with introduced in D63445. llvm-svn: 363998
* [InstSimplify] simplify power-of-2 (single bit set) sequencesSanjay Patel2019-06-201-0/+10
| | | | | | | | | | | | | | | | | | | | As discussed in PR42314: https://bugs.llvm.org/show_bug.cgi?id=42314 Improving the canonicalization for these patterns: rL363956 ...means we should adjust/enhance the related simplification. https://rise4fun.com/Alive/w1cp Name: isPow2 or zero %x = and i32 %xx, 2048 %a = add i32 %x, -1 %r = and i32 %a, %x => %r = i32 0 llvm-svn: 363997
* AMDGPU: Always use s33 for global scratch wave offsetMatt Arsenault2019-06-202-9/+1
| | | | | | | | | Every called function could possibly need this to calculate the absolute address of stack objectst, and this avoids inserting a copy around every call site in the kernel. It's also somewhat cleaner to keep this in a callee saved SGPR. llvm-svn: 363990
* [ARM GlobalISel] Add support for s64 G_ADD and G_SUB.Eli Friedman2019-06-202-2/+19
| | | | | | | | | | | | | Teach RegisterBankInfo to use the correct register class, and tell the legalizer it's legal. Everything else just works. The one thing that's slightly weird about this compared to SelectionDAG isel is that legalization can't distinguish between i64 and <1 x i64>, so we might end up with more NEON instructions than the user expects. Differential Revision: https://reviews.llvm.org/D63585 llvm-svn: 363989
* [PowerPC][NFC] Fix comments for AltVSXFMARel mapping.Jinsong Ji2019-06-201-3/+2
| | | | llvm-svn: 363987
* [profile] Solaris ld supports __start___llvm_prof_data etc. labelsRainer Orth2019-06-201-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, many profiling tests on Solaris FAIL like Command Output (stderr): -- Undefined first referenced symbol in file __llvm_profile_register_names_function /tmp/lit_tmp_Nqu4eh/infinite_loop-9dc638.o __llvm_profile_register_function /tmp/lit_tmp_Nqu4eh/infinite_loop-9dc638.o Solaris 11.4 ld supports the non-standard GNU ld extension of adding __start_SECNAME and __stop_SECNAME labels to sections whose names are valid as C identifiers. Given that we already use Solaris 11.4-only features like ld -z gnu-version-script-compat and fully working .preinit_array support in compiler-rt, we don't need to worry about older versions of Solaris ld. The patch documents that support (although the comment in lib/Transforms/Instrumentation/InstrProfiling.cpp (needsRuntimeRegistrationOfSectionRange) is quite cryptic what it's actually about), and adapts the affected testcase not to expect the alternativeq __llvm_profile_register_functions and __llvm_profile_init. It fixes all affected tests. Tested on amd64-pc-solaris2.11. Differential Revision: https://reviews.llvm.org/D41111 llvm-svn: 363984
* AMDGPU: Add intrinsics for DS GWS semaphore instructionsMatt Arsenault2019-06-205-25/+72
| | | | llvm-svn: 363983
* [LICM & MSSA] Limit unsafe sinking and hoisting.Alina Sbirlea2019-06-201-10/+50
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The getClobberingMemoryAccess API checks for clobbering accesses in a loop by walking the backedge. This may check if a memory access is being clobbered by the loop in a previous iteration, depending how smart AA got over the course of the updates in MemorySSA (it does not occur when built from scratch). If no clobbering access is found inside the loop, it will optimize to an access outside the loop. This however does not mean that access is safe to sink. Given: ``` for i load a[i] store a[i] ``` The access corresponding to the load can be optimized to outside the loop, and the load can be hoisted. But it is incorrect to sink it. In order to sink the load, we'd need to check no Def clobbers the Use in the same iteration. With this patch we currently restrict sinking to either Defs not existing in the loop, or Defs preceding the load in the same block. An easy extension is to ensure the load (Use) post-dominates all Defs. Caught by PR42294. This issue also shed light on the converse problem: hoisting stores in this same scenario would be illegal. With this patch we restrict hoisting of stores to the case when their corresponding Defs are dominating all Uses in the loop. Reviewers: george.burgess.iv Subscribers: jlebar, Prazek, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D63582 llvm-svn: 363982
* AMDGPU: Insert mem_viol check loop around GWS pre-GFX9Matt Arsenault2019-06-205-19/+129
| | | | | | | It is necessary to emit this loop around GWS operations in case the wave is preempted pre-GFX9. llvm-svn: 363979
* [InstCombine] fix typo in comment; NFCSanjay Patel2019-06-201-1/+1
| | | | llvm-svn: 363974
* [clang][NewPM] Do not eliminate available_externally durng `-O2 -flto` runsLeonard Chan2019-06-201-2/+4
| | | | | | | | | | | | This fixes CodeGen/available-externally-suppress.c when the new pass manager is turned on by default. available_externally was not emitted during -O2 -flto runs when it should still be retained for link time inlining purposes. This can be fixed by checking that we aren't LTOPrelinking when adding the EliminateAvailableExternallyPass. Differential Revision: https://reviews.llvm.org/D63580 llvm-svn: 363971
* [LFTR] Fix a (latent?) bug related to nested loopsPhilip Reames2019-06-201-0/+6
| | | | | | | | I can't actually come up with a test case this triggers on without an out of tree change, but in theory, it's a bug in the recently added multiple exit LFTR support. The root issue is that an exiting block common to two loops can (in theory) have computable exit counts for both loops. Rewriting the exit of an inner loop in terms of the outer loops IV would cause the inner loop to either a) run forever, or b) terminate on the first iteration. In practice, we appear to get lucky and not have the exit count computable for the outer loop, except when it's trivially zero. Given we bail on zero exit counts, we don't appear to ever trigger this. But I can't come up with a reason we *can't* compute an exit count for the outer loop on the common exiting block, so this may very well be triggering in some cases. llvm-svn: 363964
* [X86] Add BLSI to isUseDefConvertible.Craig Topper2019-06-201-0/+4
| | | | | | | | | | | | | | | | | | | | | Summary: BLSI sets the C flag is the input is not zero. So if its followed by a TEST of the input where only the Z flag is consumed, we can replace it with the opposite check of the C flag. We should be able to do the same for BLSMSK and BLSR, but the naive test case for those is being optimized to a subo by CodeGenPrepare. Reviewers: spatel, RKSimon Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D63589 llvm-svn: 363957
* [InstCombine] canonicalize check for power-of-2Sanjay Patel2019-06-201-0/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The form that compares against 0 is better because: 1. It removes a use of the input value. 2. It's the more standard form for this pattern: https://graphics.stanford.edu/~seander/bithacks.html#DetermineIfPowerOf2 3. It results in equal or better codegen (tested with x86, AArch64, ARM, PowerPC, MIPS). This is a root cause for PR42314, but probably doesn't completely answer the codegen request: https://bugs.llvm.org/show_bug.cgi?id=42314 Alive proof: https://rise4fun.com/Alive/9kG Name: is power-of-2 %neg = sub i32 0, %x %a = and i32 %neg, %x %r = icmp eq i32 %a, %x => %dec = add i32 %x, -1 %a2 = and i32 %dec, %x %r = icmp eq i32 %a2, 0 Name: is not power-of-2 %neg = sub i32 0, %x %a = and i32 %neg, %x %r = icmp ne i32 %a, %x => %dec = add i32 %x, -1 %a2 = and i32 %dec, %x %r = icmp ne i32 %a2, 0 llvm-svn: 363956
OpenPOWER on IntegriCloud