summaryrefslogtreecommitdiffstats
path: root/llvm/test/CodeGen/X86
Commit message (Collapse)AuthorAgeFilesLines
* Adding base lit test for x86interleavedMichael Zuckerman2017-08-241-0/+261
| | | | llvm-svn: 311658
* [x86] NFC: Clean up two tests and generate precise checks for them.Chandler Carruth2017-08-242-187/+678
| | | | | | | | | | | | Mostly this involved giving unnamed values names and running the IR through `opt` to re-format it but merging in any important comments in the original. I then deleted pointless comments and inlined the function attributes for ease of reading and editting. All of this is to make it much easier to see the instructions being generated here and evaluate any updates to the tests. llvm-svn: 311634
* [GlobalISel][X86] Support G_IMPLICIT_DEF.Igor Breger2017-08-244-0/+272
| | | | | | | | | | | | | | Summary: Support G_IMPLICIT_DEF. Reviewers: zvi, guyblank, t.p.northover Reviewed By: guyblank Subscribers: rovka, llvm-commits, kristof.beyls Differential Revision: https://reviews.llvm.org/D36733 llvm-svn: 311633
* Add ‘llvm.experimental.constrained.fma‘ Intrinsic.Wei Ding2017-08-241-22/+59
| | | | | | Differential Revision: http://reviews.llvm.org/D36335 llvm-svn: 311629
* [DAG] Fix Node Replacement in PromoteIntBinOpHans Wennborg2017-08-241-0/+53
| | | | | | | | | | | | | | When one operand is a user of another in a promoted binary operation we may replace and delete the returned value before returning triggering an assertion. Reorder node replacements to prevent this. Fixes PR34137. Landing on behalf of Nirav. Differential Revision: https://reviews.llvm.org/D36581 llvm-svn: 311623
* Parse and print DIExpressions inline to ease IR and MIR testingReid Kleckner2017-08-232-46/+48
| | | | | | | | | | | | | | | | | | | Summary: Most DIExpressions are empty or very simple. When they are complex, they tend to be unique, so checking them inline is reasonable. This also avoids the need for CodeGen passes to append to the llvm.dbg.mir named md node. See also PR22780, for making DIExpression not be an MDNode. Reviewers: aprantl, dexonsmith, dblaikie Subscribers: qcolombet, javed.absar, eraman, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D37075 llvm-svn: 311594
* [AVX512] Don't create SHRUNKBLEND SDNodes for 512-bit vectorsCraig Topper2017-08-231-0/+24
| | | | | | | | | | | | There are no 512-bit blend instructions so we shouldn't create SHRUNKBLEND for them. On a side note, it looks like there may be a missed opportunity for constant folding TESTM when LHS and RHS are equal. This fixes PR34139. Differential Revision: https://reviews.llvm.org/D36992 llvm-svn: 311572
* [XRay][CodeGen] Use PIC-friendly code in XRay sleds; remove synthetic ↵Dean Michael Berris2017-08-236-61/+51
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | references in .text Summary: This change achieves two things: - Redefine the Custom Event handling instrumentation points emitted by the compiler to not require dynamic relocation of references to the __xray_CustomEvent trampoline. - Remove the synthetic reference we emit at the end of a function that we used to keep auxiliary sections alive in favour of SHF_LINK_ORDER associated with the section where the function is defined. To achieve the custom event handling change, we've had to introduce the concept of sled versioning -- this will need to be supported by the runtime to allow us to understand how to turn on/off the new version of the custom event handling sleds. That change has to land first before we change the way we write the sleds. To remove the synthetic reference, we rely on a relatively new linker feature that preserves the sections that are associated with each other. This allows us to limit the effects on the .text section of ELF binaries. Because we're still using absolute references that are resolved at runtime for the instrumentation map (and function index) maps, we mark these sections write-able. In the future we can re-define the entries in the map to use relative relocations instead that can be statically determined by the linker. That change will be a bit more invasive so we defer this for later. Depends on D36816. Reviewers: dblaikie, echristo, pcc Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D36615 llvm-svn: 311525
* Fix tail-merge-after-mbp testMatthias Braun2017-08-231-14/+14
| | | | | | | | The output of this test changed after the fix in r311520 to have -run-pass=block-placement behave like it does in a normal pipeline. Adjust the test. llvm-svn: 311521
* [x86] auto-generate full checks; NFCSanjay Patel2017-08-221-11/+22
| | | | | | I don't see anything Darwin-specific here, so I made the target generic x86-64. llvm-svn: 311465
* [x86] simplify runs and auto-generate full checksSanjay Patel2017-08-221-33/+40
| | | | | | | | | I've replaced the two OS-specific runs with a generic run because there's no functional difference in the resulting output that we're checking. Also, the script still doesn't work with a Win target. llvm-svn: 311463
* [X86] Prevent several calls to ISD::isConstantSplatVector from returning a ↵Craig Topper2017-08-222-0/+28
| | | | | | | | | | | | | | narrower APInt than the original scalar type ISD::isConstantSplatVector can shrink to the smallest splat width. But we don't check the size of the resulting APInt at all. This can cause us to misinterpret the results. This patch just adds a flag to prevent the APInt from changing width. Fixes PR34271. Differential Revision: https://reviews.llvm.org/D36996 llvm-svn: 311429
* [X86] When selecting sse_load_f32/f64 pattern, make sure there's only one ↵Craig Topper2017-08-211-2/+3
| | | | | | | | | | | | | | use of every node all the way back to the root of the match Summary: With masked operations, its possible for the operation node like fadd, fsub, etc. to be used by multiple different vselects. Since the pattern matching will start at the vselect, we need to make sure the operation node itself is only used once before we can fold a load. Otherwise we'll end up folding the same load into multiple instructions. Reviewers: RKSimon, spatel, zvi, igorb Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D36938 llvm-svn: 311342
* [GlobalISel][X86] Support G_BRCOND operation.Igor Breger2017-08-213-0/+225
| | | | | | | | | | | | | | Summary: Support G_BRCOND operation. For now don't try to fold cmp/trunc instructions. Reviewers: zvi, guyblank Reviewed By: guyblank Subscribers: rovka, llvm-commits, kristof.beyls Differential Revision: https://reviews.llvm.org/D34754 llvm-svn: 311327
* [GlobalISel][X86] LowerCall, for now don't handel ByValue function arguments.Igor Breger2017-08-211-0/+11
| | | | llvm-svn: 311321
* [InterLeaved] Adding lit test for future work interleaved load strid 3Michael Zuckerman2017-08-211-0/+316
| | | | llvm-svn: 311320
* [x86] Teach the "generic" x86 CPU to avoid patterns that are slow onChandler Carruth2017-08-214-18/+43
| | | | | | | | | | | | | | | | | | | | widely used processors. This occured to me when I saw that we were generating 'inc' and 'dec' when for Haswell and newer we shouldn't. However, there were a few "X is slow" things that we should probably just set. I've avoided any of the "X is fast" features because most of those would be pretty serious regressions on processors where X isn't actually fast. The slow things are likely to be negligible costs on processors where these aren't slow and a significant win when they are slow. In retrospect this seems somewhat obvious. Not sure why we didn't do this a long time ago. Differential Revision: https://reviews.llvm.org/D36947 llvm-svn: 311318
* [x86] Handle more cases where we can re-use an atomic operation's flagsChandler Carruth2017-08-211-0/+86
| | | | | | | | | | | | | | | | | | | | | | rather than doing a separate comparison. This both saves an explicit comparision and avoids the use of `xadd` which introduces register constraints and other challenges to the generated code. The motivating case is from atomic reference counts where `1` is the sentinel rather than `0` for whatever reason. This can and should be lowered efficiently on x86 by just using a different flag, however the x86 code only handled the `0` case. There remains some further opportunities here that are currently hidden due to canonicalization. I've included test cases that show these and FIXMEs. However, I don't at the moment have any production use cases and they seem substantially harder to address. Differential Revision: https://reviews.llvm.org/D36945 llvm-svn: 311317
* [AVX-512] Don't change which instructions we use for unmasked subvector ↵Craig Topper2017-08-213-192/+56
| | | | | | | | | | broadcasts when AVX512DQ is enabled. There's no functional difference between the AVX512DQ instructions if we're not masking. This change unifies test checks and removes extra isel entries. Similar was done for subvector insert and extracts recently. llvm-svn: 311308
* [AVX512] Add 128->256 vbroadcastf64x2/vbroadcasti64x2 instructions to the ↵Craig Topper2017-08-212-107/+28
| | | | | | EVEX->VEX table. llvm-svn: 311307
* [AVX512] Add a test to check what happens when a load is referenced by two ↵Craig Topper2017-08-201-0/+21
| | | | | | | | different masked scalar intrinsics with the same op inputs, but different masking node. We're missing some single use checks in the sse_load_f32/f64 handling that cause us to replicate the load. llvm-svn: 311300
* [GlobalISel][X86] Support call ABI.Igor Breger2017-08-202-18/+550
| | | | | | | | | | | | | | Summary: Support call ABI. For now only Linux C and X86_64_SysV calling conventions supported. Variadic function not supported. Reviewers: zvi, guyblank, oren_ben_simhon Reviewed By: oren_ben_simhon Subscribers: rovka, kristof.beyls, llvm-commits Differential Revision: https://reviews.llvm.org/D34602 llvm-svn: 311279
* [GlobalISel][X86] Support asimetric copy from/to GPR physical register.Igor Breger2017-08-201-0/+185
| | | | | | Usually this case generated by ABI lowering, it requare to performe trancate/anyext. llvm-svn: 311278
* [x86] Fix an even stranger corner case where we have multiple levels ofChandler Carruth2017-08-191-1/+23
| | | | | | | | | cmov self-refrencing. Pointed out by Amjad Aboud in code review, test case minorly simplified from the one he posted. llvm-svn: 311267
* [AVX512] Use alignedstore256 in a pattern that's emitting a 256-bit movaps ↵Craig Topper2017-08-191-1/+1
| | | | | | from an extract subvector operation. llvm-svn: 311263
* [DAGCombiner] Extending pattern detection for vector shuffle.Jatin Bhateja2017-08-194-116/+120
| | | | | | | | | | | | | | | | Summary: If all the operands of a BUILD_VECTOR extract elements from same vector then split the vector efficiently based on the maximum vector access index. Reviewers: zvi, delena, RKSimon, thakis Reviewed By: RKSimon Subscribers: chandlerc, eladcohen, llvm-commits Differential Revision: https://reviews.llvm.org/D35788 llvm-svn: 311255
* Revert rL311247 : To rectify commit message.Jatin Bhateja2017-08-194-120/+116
| | | | | | | | Summary: This reverts commit rL311247. Differential Revision: https://reviews.llvm.org/D36927 llvm-svn: 311252
* Merge branch 'arcpatch-D35788'Jatin Bhateja2017-08-194-116/+120
| | | | llvm-svn: 311247
* Revert rL311242 "Extension of shuffle vector pattern detection, updating ↵Jatin Bhateja2017-08-193-14/+20
| | | | | | | | | | | | post rebase." Summary: This reverts commit rL311242. Differential Revision: https://reviews.llvm.org/D36924 llvm-svn: 311246
* Extension of shuffle vector pattern detection, updating post rebase.Jatin Bhateja2017-08-193-20/+14
| | | | llvm-svn: 311242
* [x86] Teach the cmov converter to aggressively convert cmovs with memoryChandler Carruth2017-08-193-2/+142
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | operands into control flow. We have seen periodically performance problems with cmov where one operand comes from memory. On modern x86 processors with strong branch predictors and speculative execution, this tends to be much better done with a branch than cmov. We routinely see cmov stalling while the load is completed rather than continuing, and if there are subsequent branches, they cannot be speculated in turn. Also, in many (even simple) cases, macro fusion causes the control flow version to be fewer uops. Consider the IACA output for the initial sequence of code in a very hot function in one of our internal benchmarks that motivates this, and notice the micro-op reduction provided. Before, SNB: ``` Throughput Analysis Report -------------------------- Block Throughput: 2.20 Cycles Throughput Bottleneck: Port1 | Num Of | Ports pressure in cycles | | | Uops | 0 - DV | 1 | 2 - D | 3 - D | 4 | 5 | | --------------------------------------------------------------------- | 1 | | 1.0 | | | | | CP | mov rcx, rdi | 0* | | | | | | | | xor edi, edi | 2^ | 0.1 | 0.6 | 0.5 0.5 | 0.5 0.5 | | 0.4 | CP | cmp byte ptr [rsi+0xf], 0xf | 1 | | | 0.5 0.5 | 0.5 0.5 | | | | mov rax, qword ptr [rsi] | 3 | 1.8 | 0.6 | | | | 0.6 | CP | cmovbe rax, rdi | 2^ | | | 0.5 0.5 | 0.5 0.5 | | 1.0 | | cmp byte ptr [rcx+0xf], 0x10 | 0F | | | | | | | | jb 0xf Total Num Of Uops: 9 ``` After, SNB: ``` Throughput Analysis Report -------------------------- Block Throughput: 2.00 Cycles Throughput Bottleneck: Port5 | Num Of | Ports pressure in cycles | | | Uops | 0 - DV | 1 | 2 - D | 3 - D | 4 | 5 | | --------------------------------------------------------------------- | 1 | 0.5 | 0.5 | | | | | | mov rax, rdi | 0* | | | | | | | | xor edi, edi | 2^ | 0.5 | 0.5 | 1.0 1.0 | | | | | cmp byte ptr [rsi+0xf], 0xf | 1 | 0.5 | 0.5 | | | | | | mov ecx, 0x0 | 1 | | | | | | 1.0 | CP | jnbe 0x39 | 2^ | | | | 1.0 1.0 | | 1.0 | CP | cmp byte ptr [rax+0xf], 0x10 | 0F | | | | | | | | jnb 0x3c Total Num Of Uops: 7 ``` The difference even manifests in a throughput cycle rate difference on Haswell. Before, HSW: ``` Throughput Analysis Report -------------------------- Block Throughput: 2.00 Cycles Throughput Bottleneck: FrontEnd | Num Of | Ports pressure in cycles | | | Uops | 0 - DV | 1 | 2 - D | 3 - D | 4 | 5 | 6 | 7 | | --------------------------------------------------------------------------------- | 0* | | | | | | | | | | mov rcx, rdi | 0* | | | | | | | | | | xor edi, edi | 2^ | | | 0.5 0.5 | 0.5 0.5 | | 1.0 | | | | cmp byte ptr [rsi+0xf], 0xf | 1 | | | 0.5 0.5 | 0.5 0.5 | | | | | | mov rax, qword ptr [rsi] | 3 | 1.0 | 1.0 | | | | | 1.0 | | | cmovbe rax, rdi | 2^ | 0.5 | | 0.5 0.5 | 0.5 0.5 | | | 0.5 | | | cmp byte ptr [rcx+0xf], 0x10 | 0F | | | | | | | | | | jb 0xf Total Num Of Uops: 8 ``` After, HSW: ``` Throughput Analysis Report -------------------------- Block Throughput: 1.50 Cycles Throughput Bottleneck: FrontEnd | Num Of | Ports pressure in cycles | | | Uops | 0 - DV | 1 | 2 - D | 3 - D | 4 | 5 | 6 | 7 | | --------------------------------------------------------------------------------- | 0* | | | | | | | | | | mov rax, rdi | 0* | | | | | | | | | | xor edi, edi | 2^ | | | 1.0 1.0 | | | 1.0 | | | | cmp byte ptr [rsi+0xf], 0xf | 1 | | 1.0 | | | | | | | | mov ecx, 0x0 | 1 | | | | | | | 1.0 | | | jnbe 0x39 | 2^ | 1.0 | | | 1.0 1.0 | | | | | | cmp byte ptr [rax+0xf], 0x10 | 0F | | | | | | | | | | jnb 0x3c Total Num Of Uops: 6 ``` Note that this cannot be usefully restricted to inner loops. Much of the hot code we see hitting this is not in an inner loop or not in a loop at all. The optimization still remains effective and indeed critical for some of our code. I have run a suite of internal benchmarks with this change. I saw a few very significant improvements and a very few minor regressions, but overall this change rarely has a significant effect. However, the improvements were very significant, and in quite important routines responsible for a great deal of our C++ CPU cycles. The gains pretty clealy outweigh the regressions for us. I also ran the test-suite and SPEC2006. Only 11 binaries changed at all and none of them showed any regressions. Amjad Aboud at Intel also ran this over their benchmarks and saw no regressions. Differential Revision: https://reviews.llvm.org/D36858 llvm-svn: 311226
* [X86][ADX] Regenerate ADX intrinsics testsSimon Pilgrim2017-08-181-45/+106
| | | | llvm-svn: 311198
* [X86][BMI2] Added scheduling test for RORX/SARX/SHLX/SHRX instructionsSimon Pilgrim2017-08-181-0/+232
| | | | llvm-svn: 311171
* [X86][AES] Add scheduling latency/throughput tests for AES instructionsSimon Pilgrim2017-08-181-0/+282
| | | | llvm-svn: 311167
* [X86][PCLMUL] Add scheduling latency/throughput test for PCLMULQDQ instructionSimon Pilgrim2017-08-181-1/+44
| | | | | | Added it to the SSE42 tests as targets seem to always have both llvm-svn: 311166
* [X86][SHA] Add scheduling latency/throughput tests for SHA instructionsSimon Pilgrim2017-08-181-0/+242
| | | | llvm-svn: 311164
* [X86][MOVBE] Add scheduling latency/throughput tests for MOVBE instructionsSimon Pilgrim2017-08-181-0/+153
| | | | llvm-svn: 311163
* [X86][BMI2] Added scheduling test for MULX instructionsSimon Pilgrim2017-08-181-0/+43
| | | | llvm-svn: 311159
* Revert "[MachineCopyPropagation] Extend pass to do COPY source forwarding" ↵Geoff Berry2017-08-1855-252/+272
| | | | | | | | | | | round 2 This reverts commit r311135. sanitizer-x86_64-linux-android buildbot is timing out with just this patch applied. llvm-svn: 311142
* Increase tail dup threshold for -O3 from 3 to 4.Richard Smith2017-08-171-1/+1
| | | | | | | | We see a modest performance improvement from this slightly higher tail dup threshold. Differential Revision: https://reviews.llvm.org/D36775 llvm-svn: 311139
* [X86] Remove SSE/AVX patterns for AND/XOR/OR/ANDN that checked for the ↵Craig Topper2017-08-171-1/+1
| | | | | | | | | | inputs being bitcasted from floating point types. There's really no reason to do this we should just let isel pick the integer version and let the execution dependency fixing pass take care of moving to FP if necessary. It's not very reliable to look for bitcasts at the edges of patterns. If for some reason one input was bitcasted and the other wasn't, or if one was a v4f32 bitcast and one was a v2f64 bitcast, we would have fallen back to the integer pattern anyway. llvm-svn: 311138
* Re-enable "[MachineCopyPropagation] Extend pass to do COPY source ↵Geoff Berry2017-08-1755-272/+252
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | forwarding" Two issues identified by buildbots were addressed: - The pass no longer forwards COPYs to physical register uses, since doing so can break code that implicitly relies on the physical register number of the use. - The pass no longer forwards COPYs to undef uses, since doing so can break the machine verifier by creating LiveRanges that don't end on a use (since the undef operand is not considered a use). [MachineCopyPropagation] Extend pass to do COPY source forwarding This change extends MachineCopyPropagation to do COPY source forwarding. This change also extends the MachineCopyPropagation pass to be able to be run during register allocation, after physical registers have been assigned, but before the virtual registers have been re-written, which allows it to remove virtual register COPY LiveIntervals that become dead through the forwarding of all of their uses. Reviewers: qcolombet, javed.absar, MatzeB, jonpa Subscribers: jyknight, nemanjai, llvm-commits, nhaehnle, mcrosier, mgorny Differential Revision: https://reviews.llvm.org/D30751 llvm-svn: 311135
* [x86] add tests for vector select-of-constants; NFCSanjay Patel2017-08-171-0/+273
| | | | | | | | We've discussed canonicalizing to this form in IR, so the backend should be prepared to lower these in ways better than what we see here in most cases. llvm-svn: 311103
* Improve line debug info when translating a CaseBlock to SDNodes.Adrian Prantl2017-08-171-0/+51
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The SelectionDAGBuilder translates various conditional branches into CaseBlocks which are then translated into SDNodes. If a conditional branch results in multiple CaseBlocks only the first CaseBlock is translated into SDNodes immediately, the rest of the CaseBlocks are put in a queue and processed when all LLVM IR instructions in the basic block have been processed. When a CaseBlock is transformed into SDNodes the SelectionDAGBuilder is queried for the current LLVM IR instruction and the resulting SDNodes are annotated with the debug info of the current instruction (if it exists and has debug metadata). When the deferred CaseBlocks are processed, the SelectionDAGBuilder does not have a current LLVM IR instruction, and the resulting SDNodes will not have any debuginfo. As DwarfDebug::beginInstruction() outputs a .loc directive for the first instruction in a labeled block (typically the case for something coming from a CaseBlock) this tends to produce a line-0 directive. This patch changes the handling of CaseBlocks to store the current instruction's debug info into the CaseBlock when it is created (and the SelectionDAGBuilder knows the current instruction) and to always use the stored debug info when translating a CaseBlock to SDNodes. Patch by Frej Drejhammar! Differential Revision: https://reviews.llvm.org/D36671 llvm-svn: 311097
* [AVX512] Don't switch unmasked subvector insert/extract instructions when ↵Craig Topper2017-08-1714-1447/+547
| | | | | | | | | | | | | | AVX512DQI is enabled. There's no reason to switch instructions with and without DQI. It just creates extra isel patterns and test divergences. There is however value in enabling the masked version of the instructions with DQI. This required introducing some new multiclasses to enabling this splitting. Differential Revision: https://reviews.llvm.org/D36661 llvm-svn: 311091
* [DAGCombiner] Add support for non-uniform constant vectors to (mul x, (1 << ↵Simon Pilgrim2017-08-171-21/+10
| | | | | | c)) -> x << c llvm-svn: 311083
* [SelectionDAG] Teach the vector-types operand scalarizer about SETCCElad Cohen2017-08-171-0/+52
| | | | | | | | | | | | | | | | When v1i1 is legal (e.g. AVX512) the legalizer can reach a case where a v1i1 SETCC with an illgeal vector type operand wasn't scalarized (since v1i1 is legal) but its operands does have to be scalarized. This used to assert because SETCC was missing from the vector operand scalarizer. This patch attemps to teach the legalizer to handle these cases by scalazring the operands, converting the node into a scalar SETCC node. Differential revision: https://reviews.llvm.org/D36651 llvm-svn: 311071
* Revert "[MachineCopyPropagation] Extend pass to do COPY source forwarding"Geoff Berry2017-08-1759-269/+294
| | | | | | | | | | This reverts commit r311038. Several buildbots are breaking, and at least one appears to be due to the forwarding of physical regs enabled by this change. Reverting while I investigate further. llvm-svn: 311062
* [x86] add cmov promotion tests for D36711; NFCSanjay Patel2017-08-161-0/+107
| | | | | | | | This way we can see what the current codegen looks like. I've also explicitly added/removed the cmov attribute from the RUN lines, so we know exactly what we're checking in the runs. llvm-svn: 311052
* [MachineCopyPropagation] Extend pass to do COPY source forwardingGeoff Berry2017-08-1659-294/+269
| | | | | | | | | | | | | | | | | | This change extends MachineCopyPropagation to do COPY source forwarding. This change also extends the MachineCopyPropagation pass to be able to be run during register allocation, after physical registers have been assigned, but before the virtual registers have been re-written, which allows it to remove virtual register COPY LiveIntervals that become dead through the forwarding of all of their uses. Reviewers: qcolombet, javed.absar, MatzeB, jonpa Subscribers: jyknight, nemanjai, llvm-commits, nhaehnle, mcrosier, mgorny Differential Revision: https://reviews.llvm.org/D30751 llvm-svn: 311038
OpenPOWER on IntegriCloud