summaryrefslogtreecommitdiffstats
path: root/llvm/test/Analysis/CostModel/X86
Commit message (Collapse)AuthorAgeFilesLines
...
* Support arbitrary address space pointers in masked gather/scatter intrinsics.Elad Cohen2017-05-032-21/+21
| | | | | | | | | | | | Fixes PR31789 - When loop-vectorize tries to use these intrinsics for a non-default address space pointer we fail with a "Calling a function with a bad singature!" assertion. This patch solves this by adding the 'vector of pointers' argument as an overloaded type which will determine the address space. Differential revision: https://reviews.llvm.org/D31490 llvm-svn: 302018
* [X86] Add missing BITREVERSE costs for SSE2 vectors and i8/i16/i32/i64 scalarsSimon Pilgrim2017-03-151-28/+24
| | | | | | Prep work for PR31810 llvm-svn: 297876
* [TargetTransformInfo] getIntrinsicInstrCost() scalarization estimation improvedJonas Paulsson2017-03-141-12/+12
| | | | | | | | | | | | | | | | | | | | | getIntrinsicInstrCost() used to only compute scalarization cost based on types. This patch improves this so that the actual arguments are checked when they are available, in order to handle only unique non-constant operands. Tests updates: Analysis/CostModel/X86/arith-fp.ll Transforms/LoopVectorize/AArch64/interleaved_cost.ll Transforms/LoopVectorize/ARM/interleaved_cost.ll The improvement in getOperandsScalarizationOverhead() to differentiate on constants made it necessary to update the interleaved_cost.ll tests even though they do not relate to intrinsics. Review: Hal Finkel https://reviews.llvm.org/D29540 llvm-svn: 297705
* [X86] Add costs for non-AVX512 single-source permutation integer shufflesMichael Kuperstein2017-02-021-20/+20
| | | | | | Differential Revision: https://reviews.llvm.org/D29416 llvm-svn: 293932
* [X86] Extend single-source shuffle cost test to test more arches. NFC.Michael Kuperstein2017-02-011-22/+129
| | | | llvm-svn: 293793
* [CostModel][X86] Fix AVX512BW vector shift costs for vXi16 typesSimon Pilgrim2017-01-153-10/+20
| | | | | | We already have patterns in place to support 128/256-bit shifts without AVX512VL llvm-svn: 292077
* [CostModel][X86] Drop separate AVX512VL checks - they match existing AVX512 ↵Simon Pilgrim2017-01-153-51/+9
| | | | | | | | costs Keep the tests though. llvm-svn: 292076
* [CostModel][X86] Update vector shift tests to correctly check by ↵Simon Pilgrim2017-01-153-199/+232
| | | | | | | | non-constant uniform values. Use shuffle( scslar_to_vector, zeroinitializer) pattern instead of shuffle( vec, zeroinitializer) llvm-svn: 292075
* [CostModel][X86] Updated vXi64 ASHR costs on AVX512 targets now that D28604 ↵Simon Pilgrim2017-01-141-8/+8
| | | | | | has landed llvm-svn: 292023
* [X86][AVX512BW] Vectorize v64i8 vector shiftsSimon Pilgrim2017-01-113-18/+18
| | | | | | Differential Revision: https://reviews.llvm.org/D28447 llvm-svn: 291665
* Fix line endingsSimon Pilgrim2017-01-114-421/+421
| | | | llvm-svn: 291663
* [X86] updating TTI costs for arithmetic instructions on X86\SLM arch.Mohammed Agabaria2017-01-111-0/+317
| | | | | | | | | | | | updated instructions: pmulld, pmullw, pmulhw, mulsd, mulps, mulpd, divss, divps, divsd, divpd, addpd and subpd. special optimization case which replaces pmulld with pmullw\pmulhw\pshuf seq. In case if the real operands bitwidth <= 16. Differential Revision: https://reviews.llvm.org/D28104 llvm-svn: 291657
* [CostModel][X86] Add AVX512VL vector shift cost tests.Simon Pilgrim2017-01-103-0/+57
| | | | llvm-svn: 291585
* [CostModel][X86] Fixed vXi8 uniform shift costs.Simon Pilgrim2017-01-085-39/+45
| | | | | | | | | | The 'fast' costs should only work for shifts by uniform constants (uniform non-constant are lowered using the slow default implementation). Logical shifts were not taking into account that we must mask the psrlw result, so the costs needed to be doubled. Added missing AVX2/AVX512BW costs as well. llvm-svn: 291391
* [CostModel][X86] Moved legal uniform shift costs earlier.Simon Pilgrim2017-01-082-8/+5
| | | | | | XOP was prematurely matching, doubling the cost of ashr/lshr uniform shifts. llvm-svn: 291390
* [CostModel][X86] Update SSE41/AVX1 vXi32 SHL costsSimon Pilgrim2017-01-071-12/+12
| | | | | | SSE41 provides pmulld which allows the simpler pslld/paddd/cvttps2dq/pmulld pattern than SSE2's use of pmuludq. llvm-svn: 291372
* [CostModel][X86] Fix AVX2 v16i16 shift 'splat' costs.Simon Pilgrim2017-01-072-12/+16
| | | | llvm-svn: 291366
* [CostModel][X86] Match 256-bit vector shift 'splat' costs for AVX2 and aboveSimon Pilgrim2017-01-073-20/+20
| | | | | | We were matching against general vector shift costs before the uniform splat costs llvm-svn: 291365
* [CostModel][AVX512BW] Add v32i16 vector shift costs for avx512bw targets.Simon Pilgrim2017-01-073-12/+12
| | | | llvm-svn: 291354
* [X86][AVX512] Use lowerShuffleAsRepeatedMaskAndLanePermute for non-VBMI ↵Simon Pilgrim2017-01-071-1/+1
| | | | | | v64i8 shuffles (PR31470) llvm-svn: 291347
* [CostModel][X86] Add AVX512 and 512-bit vector shift cost tests.Simon Pilgrim2017-01-063-18/+758
| | | | llvm-svn: 291269
* [CostModel][X86] Fix 512-bit SDIV/UDIV 'big' costs.Simon Pilgrim2017-01-061-4/+2
| | | | | | Set the costs on the lowest target that supports the type. llvm-svn: 291229
* [CostModel][X86] Add SDIV/UDIV cost tests for a wider range of targetsSimon Pilgrim2017-01-061-18/+50
| | | | | | Added a test demonstrating bug in AVX512 division costs llvm-svn: 291228
* [CostModel][X86] Include the cost of 256-bit upper subvector ↵Simon Pilgrim2017-01-051-2/+2
| | | | | | | | extract/insertion in AVX1 v4i64 MUL Matches other MUL/ADD/SUB 256-bit case on AVX1 llvm-svn: 291149
* [CostModel][X86] Add support for broadcast shuffle costsSimon Pilgrim2017-01-051-2/+138
| | | | | | | | Currently only for broadcasts with input and output of the same width. Differential Revision: https://reviews.llvm.org/D27811 llvm-svn: 291122
* [CostModel][X86] Updated vXi8 and vXi16 Reverse/Alternate shuffle costsSimon Pilgrim2017-01-041-22/+22
| | | | | | Actual codegen is much better than the extract+insert patterns that was assumed. llvm-svn: 290962
* Fixed shuffle-reverse cost on AVX-512.Elena Demikhovsky2017-01-021-1/+2
| | | | | | (This changed was approved in https://reviews.llvm.org/D28118, but Simon asked to submit it separately). llvm-svn: 290812
* AVX-512 Loop Vectorizer: Cost calculation for interleave load/store patterns.Elena Demikhovsky2017-01-029-9/+762
| | | | | | | | | | | | X86 target does not provide any target specific cost calculation for interleave patterns.It uses the common target-independent calculation, which gives very high numbers. As a result, the scalar version is chosen in many cases. The situation on AVX-512 is even worse, since we have 3-src shuffles that significantly reduce the cost. In this patch I calculate the cost on AVX-512. It will allow to compare interleave pattern with gather/scatter and choose a better solution (PR31426). * Shiffle-broadcast cost will be changed in Simon's upcoming patch. Differential Revision: https://reviews.llvm.org/D28118 llvm-svn: 290810
* [X86][SSE] Improve lowering of vXi64 multiplies Simon Pilgrim2016-12-211-24/+24
| | | | | | | | | | | | | | | | | | | | | | As mentioned on PR30845, we were performing our vXi64 multiplication as: AloBlo = pmuludq(a, b); AloBhi = pmuludq(a, psrlqi(b, 32)); AhiBlo = pmuludq(psrlqi(a, 32), b); return AloBlo + psllqi(AloBhi, 32)+ psllqi(AhiBlo, 32); when we could avoid one of the upper shifts with: AloBlo = pmuludq(a, b); AloBhi = pmuludq(a, psrlqi(b, 32)); AhiBlo = pmuludq(psrlqi(a, 32), b); return AloBlo + psllqi(AloBhi + AhiBlo, 32); This matches the lowering on gcc/icc. Differential Revision: https://reviews.llvm.org/D27756 llvm-svn: 290267
* [CostModel][X86] Updated reverse shuffle costsSimon Pilgrim2016-12-151-32/+56
| | | | llvm-svn: 289819
* [CostModel] Fix long standing bug with reverse shuffle mask detectionSimon Pilgrim2016-12-151-0/+31
| | | | | | Incorrect 'undef' mask index matching meant that broadcast shuffles could be detected as reverse shuffles llvm-svn: 289811
* [CostModel][X86] Add tests for reverse shuffle costsSimon Pilgrim2016-12-151-0/+143
| | | | llvm-svn: 289800
* [SLP] Fixed cost model for horizontal reduction.Alexey Bataev2016-12-011-1/+1
| | | | | | | | | | | | | | Currently when cost of scalar operations is evaluated the vector type is used for scalar operations. Patch fixes this issue and fixes evaluation of the vector operations cost. Several test showed that vector cost model is too optimistic. It allowed vectorization of 8 or less add/fadd operations, though scalar code is faster. Actually, only for 16 or more operations vector code provides better performance. Differential Revision: https://reviews.llvm.org/D26277 llvm-svn: 288398
* [SLP] Additional tests with the cost of vector operations.Alexey Bataev2016-12-011-0/+2
| | | | llvm-svn: 288377
* Revert "[SLP] Additional tests with the cost of vector operations."Alexey Bataev2016-12-011-2/+0
| | | | | | This reverts commit a61718435fc4118c82f8aa6133fd81f803789c1e. llvm-svn: 288371
* [SLP] Additional tests with the cost of vector operations.Alexey Bataev2016-12-011-0/+2
| | | | llvm-svn: 288369
* [X86][AVX512] Add support for v2i64 fptosi/fptoui/sitofp/uitofp on ↵Simon Pilgrim2016-11-243-9/+15
| | | | | | | | AVX512DQ-only targets Use 512-bit instructions with subvector insertion/extraction like we do in a number of similar circumstances llvm-svn: 287882
* [X86][AVX512] Add support for v4i64 fptosi/fptoui/sitofp/uitofp on ↵Simon Pilgrim2016-11-232-4/+6
| | | | | | | | AVX512DQ-only targets Use 512-bit instructions with subvector insertion/extraction like we do in a number of similar circumstances llvm-svn: 287762
* [CostModel][X86] Add missing AVX512DQ v8i64 fptosi/sitofp costsSimon Pilgrim2016-11-231-3/+3
| | | | llvm-svn: 287760
* [CostModel][X86] Add v2f32 -> v2i64 fptosi/fptoui cost testsSimon Pilgrim2016-11-232-0/+14
| | | | llvm-svn: 287756
* [CostModel][X86] Updated sitofp/uitofp scalar/vector cost testsSimon Pilgrim2016-11-222-1379/+492
| | | | | | | | Better coverage of all legal types + special cases. Removed old fptoui tests which are all handled in fptoui.ll llvm-svn: 287678
* [AVX-512] Support FCOPYSIGN for v16f32 and v8f64Craig Topper2016-11-181-2/+2
| | | | | | | | | | | | | | | Summary: This extends FCOPYSIGN support to 512-bit vectors. I've also added tests to show what the 128-bit and 256-bit cases look like with broadcast loads. Reviewers: delena, zvi, RKSimon, spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D26791 llvm-svn: 287298
* [CostModel][X86] Added mul costs for vXi8 vectorsSimon Pilgrim2016-11-141-16/+18
| | | | | | More realistic v16i8/v32i8/v64i8 MUL costs - we have to extend to vXi16, use PMULLW and then truncate the result llvm-svn: 286838
* [X86][AVX] Fixed v16i16/v32i8 ADD/SUB costs on AVX1 subtargetsSimon Pilgrim2016-11-141-8/+8
| | | | | | | | Add explicit v16i16/v32i8 ADD/SUB costs, matching the costs of v4i64/v8i32 - they were missing for some reason. This has side effects on the LV max bandwidth tests (AVX1 now prefers 128-bit vectors vs AVX2 which still prefers 256-bit) llvm-svn: 286832
* [VectorLegalizer] Expansion of CTLZ using CTPOP when possibleSimon Pilgrim2016-11-081-16/+16
| | | | | | | | | | This patch avoids scalarization of CTLZ by instead expanding to use CTPOP (ref: "Hacker's Delight") when the necessary operations are available. This also adds the necessary cost models for X86 SSE2 targets (the main beneficiary) to ensure vectorization only happens when its useful. Differential Revision: https://reviews.llvm.org/D25910 llvm-svn: 286233
* Improved cost model for FDIV and FSQRT, by Andrew TischenkoAlexey Bataev2016-10-311-82/+82
| | | | | | | | | | There is a bug describing poor cost model for floating point operations: Bug 29083 - [X86][SSE] Improve costs for floating point operations. This patch is the second one in series of patches dealing with cost model. Differential Revision: https://reviews.llvm.org/D25722 llvm-svn: 285564
* [X86][AVX512] Fix MUL v8i64 costs on non-AVX512DQ targetsSimon Pilgrim2016-10-271-2/+2
| | | | llvm-svn: 285329
* [X86][AVX512DQ] Improve lowering of MUL v2i64 and v4i64Simon Pilgrim2016-10-271-4/+13
| | | | | | | | | | With DQI but without VLX, lower v2i64 and v4i64 MUL operations with v8i64 MUL (vpmullq). Updated cost table accordingly. Differential Revision: https://reviews.llvm.org/D26011 llvm-svn: 285304
* [CostModel][X86] Added tests for current integer signed/unsigned remainder costsSimon Pilgrim2016-10-231-0/+116
| | | | llvm-svn: 284940
* [X86][SSE] Add SSE41/AVX1 costs for vector shifts.Simon Pilgrim2016-10-233-109/+109
| | | | | | We were defaulting to SSE2 costs which weren't taking into account the availability of PBLENDW/PBLENDVB to improve merging of per-element shift results. llvm-svn: 284939
OpenPOWER on IntegriCloud