summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Target/X86/X86ShuffleDecodeConstantPool.cpp
Commit message (Collapse)AuthorAgeFilesLines
* Update the file headers across all of the LLVM projects in the monorepoChandler Carruth2019-01-191-4/+3
| | | | | | | | | | | | | | | | | to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
* Recommit r344877 "[X86] Stop promoting integer loads to vXi64"Craig Topper2018-10-221-30/+28
| | | | | | | | | | | | | | | | | | | | | | | | I've included a fix to DAGCombiner::ForwardStoreValueToDirectLoad that I believe will prevent the previous miscompile. Original commit message: Theoretically this was done to simplify the amount of isel patterns that were needed. But it also meant a substantial number of our isel patterns have to match an explicit bitcast. By making the vXi32/vXi16/vXi8 types legal for loads, DAG combiner should be able to change the load type to rem I had to add some additional plain load instruction patterns and a few other special cases, but overall the isel table has reduced in size by ~12000 bytes. So it looks like this promotion was hurting us more than helping. I still have one crash in vector-trunc.ll that I'm hoping @RKSimon can help with. It seems to relate to using getTargetConstantFromNode on a load that was shrunk due to an extract_subvector combine after the constant pool entry was created. So we end up decoding more mask elements than the lo I'm hoping this patch will simplify the number of patterns needed to remove the and/or/xor promotion. Reviewers: RKSimon, spatel Reviewed By: RKSimon Subscribers: llvm-commits, RKSimon Differential Revision: https://reviews.llvm.org/D53306 llvm-svn: 344965
* Revert r344877 "[X86] Stop promoting integer loads to vXi64"Craig Topper2018-10-221-28/+30
| | | | | | Sam McCall reported miscompiles in some tensorflow code. Reverting while I try to figure out. llvm-svn: 344921
* [X86] Stop promoting integer loads to vXi64Craig Topper2018-10-211-30/+28
| | | | | | | | | | | | | | | | | | | | | Summary: Theoretically this was done to simplify the amount of isel patterns that were needed. But it also meant a substantial number of our isel patterns have to match an explicit bitcast. By making the vXi32/vXi16/vXi8 types legal for loads, DAG combiner should be able to change the load type to remove the bitcast. I had to add some additional plain load instruction patterns and a few other special cases, but overall the isel table has reduced in size by ~12000 bytes. So it looks like this promotion was hurting us more than helping. I still have one crash in vector-trunc.ll that I'm hoping @RKSimon can help with. It seems to relate to using getTargetConstantFromNode on a load that was shrunk due to an extract_subvector combine after the constant pool entry was created. So we end up decoding more mask elements than the load size. I'm hoping this patch will simplify the number of patterns needed to remove the and/or/xor promotion. Reviewers: RKSimon, spatel Reviewed By: RKSimon Subscribers: llvm-commits, RKSimon Differential Revision: https://reviews.llvm.org/D53306 llvm-svn: 344877
* Revert r344873 "foo"Craig Topper2018-10-211-28/+30
| | | | | | Rebase gone wrong left this in my tree. llvm-svn: 344875
* fooCraig Topper2018-10-211-30/+28
| | | | llvm-svn: 344873
* Remove redundant includes from lib/Target/X86.Michael Zolotukhin2017-12-131-2/+0
| | | | llvm-svn: 320636
* [APInt] Add APInt::insertBits() method to insert an APInt into a larger APIntSimon Pilgrim2017-03-101-2/+1
| | | | | | | | | | | | We currently have to insert bits via a temporary variable of the same size as the target with various shift/mask stages, resulting in further temporary variables, all of which require the allocation of memory for large APInts (MaskSizeInBits > 64). This is another of the compile time issues identified in PR32037 (see also D30265). This patch adds the APInt::insertBits() helper method which avoids the temporary memory allocation and masks/inserts the raw bits directly into the target. Differential Revision: https://reviews.llvm.org/D30780 llvm-svn: 297458
* [X86][SSE] Speed up constant pool shuffle mask decoding with direct copy ↵Simon Pilgrim2017-03-091-7/+27
| | | | | | | | (PR32037). If the constants are already the correct size, we can copy them directly into the shuffle mask. llvm-svn: 297381
* [X86] Fix SmallVector sizes in constant pool shuffle decoding to avoid heap ↵Craig Topper2017-02-271-5/+5
| | | | | | | | | | allocation Some of the vectors are under sized to avoid heap allocation. In one case the vector was oversized. Differential Revision: https://reviews.llvm.org/D30387 llvm-svn: 296353
* [X86] Use APInt instead of SmallBitVector for tracking undef elements in ↵Craig Topper2017-02-271-10/+10
| | | | | | | | | | | | | | | | | | | constant pool shuffle decoding Summary: SmallBitVector uses a malloc for more than 58 bits on a 64-bit target and more than 27 bits on a 32-bit target. Some of the vector types we deal with here use more than those number of elements and therefore cause a malloc. APInt on the other hand supports up to 64 bits without a malloc. That's the maximum number of bits we need here so we can avoid a malloc for all cases by using APInt. This will incur a minor increase in stack usage due to APInt storing the bit count separately from the data bits unlike SmallBitVector, but that should be ok. Reviewers: RKSimon Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D30386 llvm-svn: 296352
* [APInt] Add APInt::extractBits() method to extract APInt subrange (reapplied)Simon Pilgrim2017-02-251-4/+3
| | | | | | | | | | | | | | | | The current pattern for extract bits in range is typically: Mask.lshr(BitOffset).trunc(SubSizeInBits); Which can be particularly slow for large APInts (MaskSizeInBits > 64) as they require the allocation of memory for the temporary variable. This is another of the compile time issues identified in PR32037 (see also D30265). This patch adds the APInt::extractBits() helper method which avoids the temporary memory allocation. Differential Revision: https://reviews.llvm.org/D30336 llvm-svn: 296272
* Revert: r296141 [APInt] Add APInt::extractBits() method to extract APInt ↵Simon Pilgrim2017-02-241-3/+4
| | | | | | | | | | | | | | | | | | subrange The current pattern for extract bits in range is typically: Mask.lshr(BitOffset).trunc(SubSizeInBits); Which can be particularly slow for large APInts (MaskSizeInBits > 64) as they require the allocation of memory for the temporary variable. This is another of the compile time issues identified in PR32037 (see also D30265). This patch adds the APInt::extractBits() helper method which avoids the temporary memory allocation. Differential Revision: https://reviews.llvm.org/D30336 llvm-svn: 296147
* [APInt] Add APInt::extractBits() method to extract APInt subrangeSimon Pilgrim2017-02-241-4/+3
| | | | | | | | | | | | | | | | The current pattern for extract bits in range is typically: Mask.lshr(BitOffset).trunc(SubSizeInBits); Which can be particularly slow for large APInts (MaskSizeInBits > 64) as they require the allocation of memory for the temporary variable. This is another of the compile time issues identified in PR32037 (see also D30265). This patch adds the APInt::extractBits() helper method which avoids the temporary memory allocation. Differential Revision: https://reviews.llvm.org/D30336 llvm-svn: 296141
* [APInt] Add APInt::setBits() method to set all bits in rangeSimon Pilgrim2017-02-241-2/+1
| | | | | | | | | | | | | | | | | | The current pattern for setting bits in range is typically: Mask |= APInt::getBitsSet(MaskSizeInBits, LoPos, HiPos); Which can be particularly slow for large APInts (MaskSizeInBits > 64) as they require the allocation memory for the temporary variable. This is one of the key compile time issues identified in PR32037. This patch adds the APInt::setBits() helper method which avoids the temporary memory allocation completely, this first implementation uses setBit() internally instead but already significantly reduces the regression in PR32037 (~10% drop). Additional optimization may be possible. I investigated whether there is need for APInt::clearBits() and APInt::flipBits() equivalents but haven't seen these patterns to be particularly common, but reusing the code would be trivial. Differential Revision: https://reviews.llvm.org/D30265 llvm-svn: 296102
* [X86][SSE] Use APInt::getBitsSet() instead of APInt::getLowBitsSet().shl() ↵Simon Pilgrim2017-02-221-5/+6
| | | | | | separately. NFCI. llvm-svn: 295845
* Use APInt::isAllOnesValue instead of popcnt. NFCI.Simon Pilgrim2016-10-231-1/+1
| | | | | | More obvious implementation and faster too. llvm-svn: 284937
* [X86] Fix DecodeVPERMVMask to handle cases where the constant pool entry has ↵Craig Topper2016-10-181-28/+22
| | | | | | | | a different type than the shuffle itself. This is especially important for 32-bit targets with 64-bit shuffle elements. llvm-svn: 284453
* [AVX-512] Fix DecodeVPERMV3Mask to handle cases where the constant pool ↵Craig Topper2016-10-181-17/+22
| | | | | | | | | | | | | | entry has a different type than the shuffle itself. Summary: This is especially important for 32-bit targets with 64-bit shuffle elements.This is similar to how PSHUFB and VPERMIL handle the same problem. Reviewers: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D25666 llvm-svn: 284451
* [X86][SSE] Cleaned up shuffle decode assertion messagesSimon Pilgrim2016-10-011-7/+11
| | | | llvm-svn: 283050
* [X86] Avoid "unused" warnings if no assertsDouglas Katzman2016-09-291-2/+4
| | | | llvm-svn: 282732
* [X86][SSE] Added common helper for shuffle mask constant pool decodes.Simon Pilgrim2016-09-291-164/+136
| | | | | | | | | | The shuffle mask decodes have a large amount of repeated code extracting/splitting mask values from Constant data. This patch pulls all of this duplicated code into a single helper function to identify undef elements and combine/split constant integer data into the requested shuffle mask elements. Updated PSHUFB/VPERMIL/VPERMIL2/VPPERM decoders to use it (VPERMV/VPERMV3 could be converted as well in the future). llvm-svn: 282720
* [X86][AVX] Add support for target shuffle combining to VPERMILPS variable ↵Simon Pilgrim2016-07-131-3/+6
| | | | | | | | shuffle mask Added AVX512F VPERMILPS shuffle decoding support llvm-svn: 275270
* [X86][AVX512] Fixed decoding of permd/permpd variable mask shuffles + ↵Simon Pilgrim2016-07-051-3/+4
| | | | | | | | enabled them for target shuffle combining Corrected element mask masking to extract the bottom index bits (now matches the perm2 implementation but for unary inputs). llvm-svn: 274571
* Try a bit harder to remove the signed and unsigned comparison warning.Chandler Carruth2016-06-111-1/+1
| | | | | | Hopefully this time it actually works and stays away. llvm-svn: 272463
* Compare to an unsigned literal to avoid a -Wsign-compare warning.Chandler Carruth2016-06-111-1/+1
| | | | llvm-svn: 272459
* [X86][XOP] Tidied up DecodeVPERMIL2PMask to more closely match ↵Simon Pilgrim2016-06-051-3/+5
| | | | | | DecodeVPERMILPMask. llvm-svn: 271830
* [X86][XOP] Added VPERMIL2PD/VPERMIL2PS shuffle mask comment decodingSimon Pilgrim2016-06-041-0/+71
| | | | llvm-svn: 271809
* [X86][XOP] Support for VPPERM 2-input shuffle mask decodingSimon Pilgrim2016-04-091-0/+68
| | | | | | | | | | This patch adds support for decoding XOP VPPERM instruction when it represents a basic shuffle. The mask decoding required the existing MCInstrLowering code to be updated to support binary shuffles - the implementation now matches what is done in X86InstrComments.cpp. Differential Revision: http://reviews.llvm.org/D18441 llvm-svn: 265874
* [X86][AVX512] Fixed VPERMT2* shuffle mask decoding and enabled target ↵Simon Pilgrim2016-03-061-3/+4
| | | | | | | | | | | | | | shuffle combining. Patch to add support for target shuffle combining of X86ISD::VPERMV3 nodes, including support for detecting unary shuffles. This uncovered several issues with the X86ISD::VPERMV3 shuffle mask decoding of non-64 bit shuffle mask elements - the bit masking wasn't being correctly computed. Removed non-constant pool mask decode path as we have no way of testing it right now. Differential Revision: http://reviews.llvm.org/D17916 llvm-svn: 262809
* Fix spelling. NFCI.Simon Pilgrim2016-02-261-1/+1
| | | | llvm-svn: 262078
* [X86][SSE] Improve PSHUFB shuffle mask decoding.Simon Pilgrim2016-02-181-16/+36
| | | | | | | | In cases where the PSHUFB shuffle mask is shared it might not be bitcasted to a vXi8 byte vector. This patch adds support for decoding these wider shuffle masks from the ConstantPool. The test case in question makes use of this to recognise the shuffle mask is an unary UNPCKL pattern and simplifies accordingly. llvm-svn: 261201
* [X86] Move shuffle decoding for constant pool into the X86CodeGen library to ↵Craig Topper2015-12-311-0/+190
remove a layering violation in the Util library. llvm-svn: 256680
OpenPOWER on IntegriCloud