summaryrefslogtreecommitdiffstats
path: root/llvm/lib/IR/AutoUpgrade.cpp
Commit message (Collapse)AuthorAgeFilesLines
* [x86] Fix a bug in the auto-upgrade from r276416 where we failed to giveChandler Carruth2016-08-101-1/+1
| | | | | | | | | | | | | | | | | | | a sufficiently low alignment for the IR load created. There is no test case because we don't have any test cases for the *IR* produced by the autoupgrade, only the x86 assembly, and it happens that the x86 assembly for this intrinsic as it is tested in the autoupgrade path just happens to not produce a separate load instruction where we might have observed the alignment. I'm going to follow up on the original commit to suggest getting IR-level testing in addition to the asm level testing here so that we can see and test these kinds of issues. We might never get an x86 instruction out with an alignment constraint, but we could stil miscompile code by folding against the alignment marked on (or inferred for in this case) the load. llvm-svn: 278203
* Invariant start/end intrinsics overloaded for address spaceAnna Thomas2016-07-221-1/+27
| | | | | | | | | | | | | | | | | | | | | | Summary: The llvm.invariant.start and llvm.invariant.end intrinsics currently support specifying invariant memory objects only in the default address space. With this change, these intrinsics are overloaded for any adddress space for memory objects and we can use these llvm invariant intrinsics in non-default address spaces. Example: llvm.invariant.start.p1i8(i64 4, i8 addrspace(1)* %ptr) This overloaded intrinsic is needed for representing final or invariant memory in managed languages. Reviewers: apilipenko, reames Subscribers: llvm-commits llvm-svn: 276447
* [X86][AVX] Added support for lowering to VBROADCASTF128/VBROADCASTI128 ↵Simon Pilgrim2016-07-221-7/+14
| | | | | | | | | | | | | | | | (reapplied) As reported on PR26235, we don't currently make use of the VBROADCASTF128/VBROADCASTI128 instructions (or the AVX512 equivalents) to load+splat a 128-bit vector to both lanes of a 256-bit vector. This patch enables lowering from subvector insertion/concatenation patterns and auto-upgrades the llvm.x86.avx.vbroadcastf128.pd.256 / llvm.x86.avx.vbroadcastf128.ps.256 intrinsics to match. We could possibly investigate using VBROADCASTF128/VBROADCASTI128 to load repeated constants as well (similar to how we already do for scalar broadcasts). Reapplied with fix for PR28657 - removed intrinsic definitions (clang companion patch to be be submitted shortly). Differential Revision: https://reviews.llvm.org/D22460 llvm-svn: 276416
* Revert "[X86][AVX] Added support for lowering to VBROADCASTF128/VBROADCASTI128"Benjamin Kramer2016-07-221-14/+7
| | | | | | | | It caused PR28657. This reverts commit r276281. llvm-svn: 276405
* Revert "Invariant start/end intrinsics overloaded for address space"Anna Thomas2016-07-211-27/+1
| | | | | | This reverts commit r276316. llvm-svn: 276320
* Invariant start/end intrinsics overloaded for address spaceAnna Thomas2016-07-211-1/+27
| | | | | | | | | | | | | | | | | | | | | Summary: The llvm.invariant.start and llvm.invariant.end intrinsics currently support specifying invariant memory objects only in the default address space. With this change, these intrinsics are overloaded for any adddress space for memory objects and we can use these llvm invariant intrinsics in non-default address spaces. Example: llvm.invariant.start.p1i8(i64 4, i8 addrspace(1)* %ptr) This overloaded intrinsic is needed for representing final or invariant memory in managed languages. Reviewers: tstellarAMD, reames, apilipenko Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D22519 llvm-svn: 276316
* [X86][AVX] Added support for lowering to VBROADCASTF128/VBROADCASTI128Simon Pilgrim2016-07-211-7/+14
| | | | | | | | | | | | As reported on PR26235, we don't currently make use of the VBROADCASTF128/VBROADCASTI128 instructions (or the AVX512 equivalents) to load+splat a 128-bit vector to both lanes of a 256-bit vector. This patch enables lowering from subvector insertion/concatenation patterns and auto-upgrades the llvm.x86.avx.vbroadcastf128.pd.256 / llvm.x86.avx.vbroadcastf128.ps.256 intrinsics to match. We could possibly investigate using VBROADCASTF128/VBROADCASTI128 to load repeated constants as well (similar to how we already do for scalar broadcasts). Differential Revision: https://reviews.llvm.org/D22460 llvm-svn: 276281
* [X86][SSE] Reimplement SSE fp2si conversion intrinsics instead of using ↵Simon Pilgrim2016-07-191-8/+0
| | | | | | | | | | | | | | | | | | generic IR D20859 and D20860 attempted to replace the SSE (V)CVTTPS2DQ and VCVTTPD2DQ truncating conversions with generic IR instead. It turns out that the behaviour of these intrinsics is different enough from generic IR that this will cause problems, INF/NAN/out of range values are guaranteed to result in a 0x80000000 value - which plays havoc with constant folding which converts them to either zero or UNDEF. This is also an issue with the scalar implementations (which were already generic IR and what I was trying to match). This patch changes both scalar and packed versions back to using x86-specific builtins. It also deals with the other scalar conversion cases that are runtime rounding mode dependent and can have similar issues with constant folding. A companion clang patch is at D22105 Differential Revision: https://reviews.llvm.org/D22106 llvm-svn: 275981
* [AVX512] Remove masked logic op intrinsics and autoupgrade them to native IR.Craig Topper2016-07-121-0/+21
| | | | llvm-svn: 275155
* [X86,IR] Remove unnecessary or unused LLVMContext parameter from some of the ↵Craig Topper2016-07-121-17/+16
| | | | | | X86 intrinsic upgrade functions. llvm-svn: 275138
* [X86] Remove and autoupgrade 512-bit non-temporal store intrinsics.Craig Topper2016-07-091-2/+6
| | | | llvm-svn: 274966
* Move setName after accessing NameEric Liu2016-07-081-5/+2
| | | | llvm-svn: 274862
* Make a std::string copy of StringRef Name so that it remains valid when the ↵Eric Liu2016-07-081-1/+4
| | | | | | | | | | | | | | original Name is overridden. Summary: lib/IR/AutoUpgrade.cpp:348 and lib/IR/AutoUpgrade.cpp:350 upset sanitizer. Reviewers: bkramer Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D22140 llvm-svn: 274861
* [AVX512] Remove and autoupgrade a duplicate set of 512-bit masked shift ↵Craig Topper2016-07-081-1/+24
| | | | | | | | intrinsics. I'm not sure if clang ever used these builtin names or not. llvm-svn: 274827
* [AVX512] Zero extend the result of vpcmpeq/vpcmpgt and similar intrinsics in ↵Craig Topper2016-07-071-2/+3
| | | | | | the autoupgrade code. This currently results in worse codegen but is needed for correctness. llvm-svn: 274736
* [X86][AVX512] Autoupgrade the BROADCAST intrinsicsSimon Pilgrim2016-07-051-1/+9
| | | | llvm-svn: 274550
* [IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so ↵Craig Topper2016-07-041-214/+226
| | | | | | we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
* [X86][AVX512] Autoupgrade the VPERMPD/VPERMQ intrinsicsSimon Pilgrim2016-07-041-0/+18
| | | | llvm-svn: 274506
* [X86][AVX512] Autoupgrade the VPERMILPD/VPERMILPS intrinsicsSimon Pilgrim2016-07-041-1/+3
| | | | llvm-svn: 274498
* [X86][AVX512] Autoupgrade the MOVDDUP/MOVSLDUP/MOVSHDUP intrinsicsSimon Pilgrim2016-07-021-0/+25
| | | | llvm-svn: 274439
* Support arbitrary addrspace pointers in masked load/store intrinsicsArtur Pilipenko2016-06-281-0/+34
| | | | | | | | | | | | | | This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
* Revert -r273892 "Support arbitrary addrspace pointers in masked load/store ↵Artur Pilipenko2016-06-271-34/+0
| | | | | | intrinsics" since some of the clang tests don't expect to see the updated signatures. llvm-svn: 273895
* Support arbitrary addrspace pointers in masked load/store intrinsicsArtur Pilipenko2016-06-271-0/+34
| | | | | | | | | | | | | | This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 273892
* [AVX512] Remove masked unpack intrinsics and autoupgrade to vectorshuffle ↵Craig Topper2016-06-231-0/+36
| | | | | | and selects. llvm-svn: 273543
* [AVX512] Remove the masked vpcmpeq/vcmpgt intrinsics and autoupgrade them to ↵Craig Topper2016-06-211-0/+30
| | | | | | native icmps. llvm-svn: 273240
* [X86][SSE4A] Autoupgrade and remove MOVNTSD/MOVNTSS intrinsicsSimon Pilgrim2016-06-181-0/+25
| | | | | | Required better annotation of the instruction defs upon removal of the builtin intrinsic pattern. llvm-svn: 273077
* [x86] autoupgrade and remove AVX2 integer min/max intrinsicsSanjay Patel2016-06-161-4/+10
| | | | | | | This will (hopefully very temporarily) break clang. The clang side of this should be the next commit. llvm-svn: 272932
* [x86] autoupgrade and remove SSE2/SSE41 integer min/max intrinsicsSanjay Patel2016-06-161-0/+36
| | | | | | | | Follow-up to: http://reviews.llvm.org/rL272806 http://reviews.llvm.org/rL272807 llvm-svn: 272907
* fix comments; NFCSanjay Patel2016-06-151-6/+3
| | | | llvm-svn: 272848
* [AVX512] Remove maksed pshufd, pshuflw, and phufhw intrinsics and ↵Craig Topper2016-06-131-3/+21
| | | | | | autoupgrade them to selects and shufflevector. llvm-svn: 272527
* [X86] Refactor some of the X86 autoupgrade code to split mask vector and ↵Craig Topper2016-06-131-74/+69
| | | | | | select generation into routines that can be reused for future intrinsic upgrades. NFC llvm-svn: 272526
* [X86] Remove sse2 pshufd/pshuflw/pshufhw intrinsics and upgrade them to ↵Craig Topper2016-06-121-1/+31
| | | | | | shufflevector. llvm-svn: 272510
* [X86] Pre-allocate some of the shuffle mask SmallVectors in the auto upgrade ↵Craig Topper2016-06-121-20/+12
| | | | | | code instead of calling push_back in a loop. This removes the need to check if the vector needs to grow on each iteration. llvm-svn: 272501
* [X86] Greatly simplify the llvm.x86.avx.vpermil.* auto-upgrade code. We can ↵Craig Topper2016-06-121-36/+18
| | | | | | fully derive everything using types of the intrinsic arguments rather than writing separate loops for each intrinsic. NFC llvm-svn: 272496
* [X86,IR] Make use of the CreateShuffleVector form that takes an ↵Craig Topper2016-06-121-20/+19
| | | | | | ArrayRef<uint32_t> to avoid the need to manually create a bunch of Constants and a ConstantVector. NFC llvm-svn: 272493
* [IR] Require ArrayRef of 'uint32_t' instead of 'int' for the mask argument ↵Craig Topper2016-06-121-10/+11
| | | | | | for one of the signatures of CreateShuffleVector. This better emphasises that you can't use it for the -1 as undef behavior. llvm-svn: 272491
* [X86][AVX512] Dropped avx512 VPSLLDQ/VPSRLDQ intrinsicsSimon Pilgrim2016-06-091-10/+14
| | | | | | Auto-upgrade to generic shuffles like sse/avx2 implementations now that we can lower to VPSLLDQ/VPSRLDQ llvm-svn: 272308
* [AVX512] Remove masked palignr intrinsics and auto-upgrade them to native IR ↵Craig Topper2016-06-061-0/+54
| | | | | | of vector shuffle and select. llvm-svn: 271872
* [X86][XOP] Support for VPERMIL2PD/VPERMIL2PS 2-input shuffle instructionsSimon Pilgrim2016-06-031-0/+35
| | | | | | | | | | | | This patch begins adding support for lowering to the XOP VPERMIL2PD/VPERMIL2PS shuffle instructions - adding the X86ISD::VPERMIL2 opcode and cleaning up the usage. The internal llvm intrinsics were assuming the shuffle mask operand was the same type as the float/double input operands (I guess to simplify the intrinsic definitions in X86InstrXOP.td to a single value type). These needed changing to integer types (matching the clang builtin and the AMD intrinsics definitions), an auto upgrade path is added to convert old calls. Mask decoding/target shuffle support will be added in future patches. Differential Revision: http://reviews.llvm.org/D20049 llvm-svn: 271633
* [X86][SSE] Replace (V)CVTTPS2DQ and VCVTTPD2DQ truncating (round to zero) ↵Simon Pilgrim2016-06-021-0/+8
| | | | | | | | | | | | f32/f64 to i32 with generic IR (llvm) This patch removes the llvm intrinsics (V)CVTTPS2DQ and VCVTTPD2DQ truncation (round to zero) conversions and auto-upgrades to FP_TO_SINT calls instead. Note: I looked at updating CVTTPD2DQ as well but this still requires a lot more work to correctly lower. Differential Revision: http://reviews.llvm.org/D20860 llvm-svn: 271510
* [AVX512] Remove masked load intrinsics. Clang now emits generic masked load ↵Craig Topper2016-06-021-2/+62
| | | | | | | | intrinsics instead. The intrinsics will be autoupgraded to the same generic masked loads. llvm-svn: 271478
* Revert r271362 "[AVX512] Remove masked load intrinsics. Clang now emits ↵Craig Topper2016-06-011-62/+2
| | | | | | | | generic masked load intrinsics instead." Looks like something isn't quite right still. Also forgot to move the test cases to an autoupgrade test. llvm-svn: 271363
* [AVX512] Remove masked load intrinsics. Clang now emits generic masked load ↵Craig Topper2016-06-011-2/+62
| | | | | | | | intrinsics instead. The intrinsics will be autoupgraded to the same generic masked loads. llvm-svn: 271362
* [AVX512] Remove masked store intrinsics. Clang now emits generic masked ↵Craig Topper2016-05-311-0/+66
| | | | | | | | store intrinsics instead. The intrinsics will be autoupgraded to the same generic masked stores. llvm-svn: 271245
* [X86] Remove SSE/AVX unaligned store intrinsics as clang no longer uses ↵Craig Topper2016-05-301-0/+17
| | | | | | them. Auto upgrade to native unaligned store instructions. llvm-svn: 271236
* [X86] Simplify and remove some unnecessary auto-upgrade code.Craig Topper2016-05-301-10/+2
| | | | llvm-svn: 271233
* [X86] Reduce the number of string compares in the autoupgrade logic by ↵Craig Topper2016-05-301-41/+17
| | | | | | checking more prefixes instead of complete matches. llvm-svn: 271232
* [X86] Simplify some of the autoupgrade code. NFCCraig Topper2016-05-291-61/+38
| | | | llvm-svn: 271174
* [X86][SSE] (Reapplied) Replace (V)PMOVSX and (V)PMOVZX integer extension ↵Simon Pilgrim2016-05-281-3/+12
| | | | | | | | | | | | intrinsics with generic IR (llvm) This patch removes the llvm intrinsics VPMOVSX and (V)PMOVZX sign/zero extension intrinsics and auto-upgrades to SEXT/ZEXT calls instead. We already did this for SSE41 PMOVSX sometime ago so much of that implementation can be reused. Reapplied now that the the companion patch (D20684) removes/auto-upgrade the clang intrinsics has been committed. Differential Revision: http://reviews.llvm.org/D20686 llvm-svn: 271131
* Revert: r270973 - [X86][SSE] Replace (V)PMOVSX and (V)PMOVZX integer ↵Simon Pilgrim2016-05-271-12/+3
| | | | | | extension intrinsics with generic IR (llvm) llvm-svn: 270976
OpenPOWER on IntegriCloud