summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Target/X86/X86ExpandPseudo.cpp
Commit message (Collapse)AuthorAgeFilesLines
* [CodeGen] Make branch funnels pass the machine verifierFrancis Visoiu Mistrih2019-07-031-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We previously marked all the tests with branch funnels as `-verify-machineinstrs=0`. This is an attempt to fix it. 1) `ICALL_BRANCH_FUNNEL` has no defs. Mark it as `let OutOperandList = (outs)` 2) After that we hit an assert: ``` Assertion failed: (Op.getValueType() != MVT::Other && Op.getValueType() != MVT::Glue && "Chain and glue operands should occur at end of operand list!"), function AddOperand, file /Users/francisvm/llvm/llvm/lib/CodeGen/SelectionDAG/InstrEmitter.cpp, line 461. ``` The chain operand was added at the beginning of the operand list. Move that to the end. 3) After that we hit another verifier issue in the pseudo expansion where the registers used in the cmps and jmps are not added to the livein lists. Add the `EFLAGS` to all the new MBBs that we create. PR39436 Differential Review: https://reviews.llvm.org/D54155 llvm-svn: 365058
* [Backend] Keep call site info valid through the backendDjordje Todorovic2019-06-271-0/+1
| | | | | | | | | | | | | | | | | | | | Handle call instruction replacements and deletions in order to preserve valid state of the call site info of the MachineFunction. NOTE: If the call site info is enabled for a new target, the assertion from the MachineFunction::DeleteMachineInstr() should help to locate places where the updateCallSiteInfo() should be called in order to preserve valid state of the call site info. ([10/13] Introduce the debug entry values.) Co-authored-by: Ananth Sowda <asowda@cisco.com> Co-authored-by: Nikola Prica <nikola.prica@rt-rk.com> Co-authored-by: Ivan Baev <ibaev@cisco.com> Differential Revision: https://reviews.llvm.org/D61062 llvm-svn: 364536
* [X86] Preserve operand flag when expanding TCRETURNriFrancis Visoiu Mistrih2019-04-051-2/+9
| | | | | | | | | The expansion of TCRETURNri(64) would not keep operand flags like undef/renamable/etc. which can result in machine verifier issues. Also add plumbing to be able to use `-run-pass=x86-pseudo`. llvm-svn: 357808
* [X86] Merge the different Jcc instructions for each condition code into ↵Craig Topper2019-04-051-9/+9
| | | | | | | | | | | | | | | | | | | | | single instructions that store the condition code as an operand. Summary: This avoids needing an isel pattern for each condition code. And it removes translation switches for converting between Jcc instructions and condition codes. Now the printer, encoder and disassembler take care of converting the immediate. We use InstAliases to handle the assembly matching. But we print using the asm string in the instruction definition. The instruction itself is marked IsCodeGenOnly=1 to hide it from the assembly parser. Reviewers: spatel, lebedev.ri, courbet, gchatelet, RKSimon Reviewed By: RKSimon Subscribers: MatzeB, qcolombet, eraman, hiraditya, arphaman, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D60228 llvm-svn: 357802
* Update the file headers across all of the LLVM projects in the monorepoChandler Carruth2019-01-191-4/+3
| | | | | | | | | | | | | | | | | to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
* Use branch funnels for virtual calls when retpoline mitigation is enabled.Peter Collingbourne2018-03-091-0/+103
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The retpoline mitigation for variant 2 of CVE-2017-5715 inhibits the branch predictor, and as a result it can lead to a measurable loss of performance. We can reduce the performance impact of retpolined virtual calls by replacing them with a special construct known as a branch funnel, which is an instruction sequence that implements virtual calls to a set of known targets using a binary tree of direct branches. This allows the processor to speculately execute valid implementations of the virtual function without allowing for speculative execution of of calls to arbitrary addresses. This patch extends the whole-program devirtualization pass to replace certain virtual calls with calls to branch funnels, which are represented using a new llvm.icall.jumptable intrinsic. It also extends the LowerTypeTests pass to recognize the new intrinsic, generate code for the branch funnels (x86_64 only for now) and lay out virtual tables as required for each branch funnel. The implementation supports full LTO as well as ThinLTO, and extends the ThinLTO summary format used for whole-program devirtualization to support branch funnels. For more details see RFC: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120672.html Differential Revision: https://reviews.llvm.org/D42453 llvm-svn: 327163
* [DWARF] Fix incorrect prologue end line record.Paul Robinson2018-02-141-3/+3
| | | | | | | | | | | | The prologue-end line record must be emitted after the last instruction that is part of the function frame setup code and before the instruction that marks the beginning of the function body. Patch by Carlos Alberto Enciso! Differential Revision: https://reviews.llvm.org/D41762 llvm-svn: 325143
* MachineFunction: Return reference from getFunction(); NFCMatthias Braun2017-12-151-1/+1
| | | | | | The Function can never be nullptr so we can return a reference. llvm-svn: 320884
* Re-apply r282920 "X86: Allow conditional tail calls in Win64 "leaf" ↵Hans Wennborg2017-02-161-2/+3
| | | | | | | | | | functions (PR26302)" The original commit was reverted in r283329 due to a miscompile in Chromium. That turned out to be the same issue as PR31257, which was fixed in r295262. llvm-svn: 295357
* [X86] Re-enable conditional tail calls and fix PR31257.Hans Wennborg2017-02-161-1/+20
| | | | | | | | | | | This reverts r294348, which removed support for conditional tail calls due to the PR above. It fixes the PR by marking live registers as implicitly used and defined by the now predicated tailcall. This is similar to how IfConversion predicates instructions. Differential Revision: https://reviews.llvm.org/D29856 llvm-svn: 295262
* [X86] Disable conditional tail calls (PR31257)Hans Wennborg2017-02-071-20/+1
| | | | | | | | | They are currently modelled incorrectly (as calls, which clobber registers, confusing e.g. Machine Copy Propagation). Reverting until we figure out the proper solution. llvm-svn: 294348
* [CodeGen] Rename MachineInstrBuilder::addOperand. NFCDiana Picus2017-01-131-2/+2
| | | | | | | | | | | Rename from addOperand to just add, to match the other method that has been added to MachineInstrBuilder for adding more than just 1 operand. See https://reviews.llvm.org/D28057 for the whole discussion. Differential Revision: https://reviews.llvm.org/D28556 llvm-svn: 291891
* Revert r282920 "X86: Allow conditional tail calls in Win64 "leaf" functions ↵Hans Wennborg2016-10-051-3/+2
| | | | | | | | | (PR26302)" This is suspected to cause a miscompile in Chromium. Reverting while investigating. llvm-svn: 283329
* Use StringRef in Pass/PassManager APIs (NFC)Mehdi Amini2016-10-011-1/+1
| | | | llvm-svn: 283004
* X86: Allow conditional tail calls in Win64 "leaf" functions (PR26302)Hans Wennborg2016-09-301-2/+3
| | | | | | | | | | | We can't use Jcc to leave a Win64 function in general, because that confuses the unwinder. However, for "leaf" functions, that is, functions where the return address is always on top of the stack and which don't have unwind info, it's OK. Differential Revision: https://reviews.llvm.org/D24836 llvm-svn: 282920
* X86: Fold tail calls into conditional branches also for 64-bit (PR26302)Hans Wennborg2016-09-091-3/+9
| | | | | | | | | This extends the optimization in r280832 to also work for 64-bit. The only quirk is that we can't do this for 64-bit Windows (yet). Differential Revision: https://reviews.llvm.org/D24423 llvm-svn: 281113
* Win64: Don't use REX prefix for direct tail callsHans Wennborg2016-09-081-1/+3
| | | | | | | | | | The REX prefix should be used on indirect jmps, but not direct ones. For direct jumps, the unwinder looks at the offset to determine if it's inside the current function. Differential Revision: https://reviews.llvm.org/D24359 llvm-svn: 281003
* X86: Fold tail calls into conditional branches where possible (PR26302)Hans Wennborg2016-09-071-7/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When branching to a block that immediately tail calls, it is possible to fold the call directly into the branch if the call is direct and there is no stack adjustment, saving one byte. Example: define void @f(i32 %x, i32 %y) { entry: %p = icmp eq i32 %x, %y br i1 %p, label %bb1, label %bb2 bb1: tail call void @foo() ret void bb2: tail call void @bar() ret void } before: f: movl 4(%esp), %eax cmpl 8(%esp), %eax jne .LBB0_2 jmp foo .LBB0_2: jmp bar after: f: movl 4(%esp), %eax cmpl 8(%esp), %eax jne bar .LBB0_1: jmp foo I don't expect any significant size savings from this (on a Clang bootstrap I saw 288 bytes), but it does make the code a little tighter. This patch only does 32-bit, but 64-bit would work similarly. Differential Revision: https://reviews.llvm.org/D24108 llvm-svn: 280832
* MachineFunctionProperties/MIRParser: Rename AllVRegsAllocated->NoVRegs, ↵Matthias Braun2016-08-251-1/+1
| | | | | | | | | | | | | compute it Rename AllVRegsAllocated to NoVRegs. This avoids the connotation of running after register and simply describes that no vregs are used in a machine function. With that we can simply compute the property and do not need to dump/parse it in .mir files. Differential Revision: http://reviews.llvm.org/D23850 llvm-svn: 279698
* X86: Avoid implicit iterator conversions, NFCDuncan P. N. Exon Smith2016-07-121-2/+2
| | | | | | | | Avoid implicit conversions from MachineInstrBundleIterator to MachineInstr*, mainly by preferring MachineInstr& over MachineInstr* and using range-based for loops. llvm-svn: 275149
* [X86] Fix tailcall return address clobber bug.Quentin Colombet2016-07-111-3/+12
| | | | | | | | | | | | | | | | | | | | | | | | This bug (llvm.org/PR28124) was introduced by r237977, which refactored the tail call sequence to be generated in two passes instead of one. Unfortunately, the stack adjustment produced by the first pass was not recognized by X86FrameLowering::mergeSPUpdates() in all cases, causing code such as the following, which clobbers the return address, to be generated: popl %edi popl %edi pushl %eax jmp tailcallee # TAILCALL To fix the problem, the entire stack adjustment is performed in X86ExpandPseudo::ExpandMI() for tail calls. Patch by Magnus Lång <margnus1@gmail.com> Differential Revision: http://reviews.llvm.org/D21325 llvm-svn: 275103
* Add MachineFunctionProperty checks for AllVRegsAllocated for target passesDerek Schuff2016-04-041-0/+5
| | | | | | | | | | | | | | Summary: This adds the same checks that were added in r264593 to all target-specific passes that run after register allocation. Reviewers: qcolombet Subscribers: jyknight, dsanders, llvm-commits Differential Revision: http://reviews.llvm.org/D18525 llvm-svn: 265313
* [X86] Make sure we do not clobber RBX with cmpxchg when used as a base pointer.Quentin Colombet2016-03-121-0/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | cmpxchg[8|16]b uses RBX as one of its argument. In other words, using this instruction clobbers RBX as it is defined to hold one the input. When the backend uses dynamically allocated stack, RBX is used as a reserved register for the base pointer. Reserved registers have special semantic that only the target understands and enforces, because of that, the register allocator don’t use them, but also, don’t try to make sure they are used properly (remember it does not know how they are supposed to be used). Therefore, when RBX is used as a reserved register but defined by something that is not compatible with that use, the register allocator will not fix the surrounding code to make sure it gets saved and restored properly around the broken code. This is the responsibility of the target to do the right thing with its reserved register. To fix that, when the base pointer needs to be preserved, we use a different pseudo instruction for cmpxchg that save rbx. That pseudo takes two more arguments than the regular instruction: - One is the value to be copied into RBX to set the proper value for the comparison. - The other is the virtual register holding the save of the value of RBX as the base pointer. This saving is done as part of isel (i.e., we emit a copy from rbx). cmpxchg_save_rbx <regular cmpxchg args>, input_for_rbx_reg, save_of_rbx_as_bp This gets expanded into: rbx = copy input_for_rbx_reg cmpxchg <regular cmpxchg args> rbx = save_of_rbx_as_bp Note: The actual modeling of the pseudo is a bit more complicated to make sure the interferes that appears after the pseudo gets expanded are properly modeled before that expansion. This fixes PR26883. llvm-svn: 263325
* Fix build breakageDavid Majnemer2016-03-041-4/+5
| | | | llvm-svn: 262756
* [X86] Support cleaning more than 2**16 bytes of stackDavid Majnemer2016-03-041-0/+25
| | | | | | | | | | | | | | | | | | | The x86 ret instruction has a 16 bit immediate indicating how many bytes to pop off of the stack beyond the return address. There is a problem when extremely large structs are passed by value: we might not be able to fit the number of bytes to pop into the return instruction. To fix this, expand RET_FLAG a little later and use a special sequence to clean the stack: pop %ecx ; return address is now in %ecx add $n, %esp ; clean the stack push %ecx ; bring the return address back on the stack ret ; pop the return address and jmp to it's value llvm-svn: 262755
* CodeGen: Change MachineInstr to use MachineInstr&, NFCDuncan P. N. Exon Smith2016-02-271-1/+1
| | | | | | | | Change MachineInstr API to prefer MachineInstr& over MachineInstr* whenever the parameter is expected to be non-null. Slowly inching toward being able to fix PR26753. llvm-svn: 262149
* Fix line endings after r256155. NFC.Eric Christopher2015-12-211-198/+198
| | | | llvm-svn: 256211
* Implemented Support of IA interrupt and exception handlers:Amjad Aboud2015-12-211-189/+198
| | | | | | | | http://lists.llvm.org/pipermail/cfe-dev/2015-September/045171.html Differential Revision: http://reviews.llvm.org/D15567 llvm-svn: 256155
* Move EH-specific helper functions to a more appropriate placeDavid Majnemer2015-12-021-1/+1
| | | | | | No functionality change is intended. llvm-svn: 254562
* [WinEH] Fix ESP management with 32-bit __CxxFrameHandler3Reid Kleckner2015-11-131-1/+4
| | | | | | | | | | | | | | | The C++ EH personality automatically restores ESP from the C++ EH registration node after a catchret. I mistakenly thought it was like SEH, which does not restore ESP. It makes sense for C++ EH to differ from SEH here because SEH does not use funclets for catches, and does not allow catching inside of finally. C++ EH may need to unwind through multiple catch funclets and eventually catchret to some outer funclet. Therefore, the runtime has to keep track of which ESP to use with catchret, rather than having the compiler reload it manually. llvm-svn: 253084
* [WinEH] Split EH_RESTORE out of CATCHRET for 32-bit EHReid Kleckner2015-11-061-0/+7
| | | | | | | | | | | | | | | | | | | | | | | This adds the EH_RESTORE x86 pseudo instr, which is responsible for restoring the stack pointers: EBP and ESP, and ESI if stack realignment is involved. We only need this on 32-bit x86, because on x64 the runtime restores CSRs for us. Previously we had to keep the CATCHRET instruction around during SEH so that we could convince X86FrameLowering to restore our frame pointers. Now we can split these instructions earlier. This was confusing, because we had a return instruction which wasn't really a return and was ultimately going to be removed by X86FrameLowering. This change also simplifies X86FrameLowering, which really shouldn't be building new MBBs. No observable functional change currently, but with the new register mask stuff in D14407, CATCHRET will become a register allocator barrier, and our existing tests rely on us having reasonable register allocation around SEH. llvm-svn: 252266
* [WinEH] Ensure that funclets obey the x64 ABIDavid Majnemer2015-09-291-35/+0
| | | | | | | | | | The x64 ABI requires that epilogues do not contain code other than stack adjustments and some limited control flow. However, we'd insert code to initialize the return address after stack adjustments. Instead, insert EAX/RAX with the current value before we create the stack adjustments in the epilogue. llvm-svn: 248839
* [WinEH] Make funclet return instrs pseudo instrsReid Kleckner2015-09-171-0/+35
| | | | | | | | | This makes catchret look more like a branch, and less like a weird use of BlockAddress. It also lets us get away from llvm.x86.seh.restoreframe, which relies on the old parentfpoffset label arithmetic. llvm-svn: 247936
* [X86] Refactor stack adjustments into X86FrameLowering::BuildStackAdjustmentReid Kleckner2015-06-181-7/+2
| | | | | | | | Deduplicates some code and lets us use LEA on atom when adjusting the stack around callee-cleanup calls. This is the only intended functionality change. llvm-svn: 240044
* [X86] Remove unneeded parameters and deduplicate stack alignment codeReid Kleckner2015-06-181-7/+2
| | | | | | NFC llvm-svn: 240033
* Re-land "[X86] Cache variables that only depend on the subtarget"Reid Kleckner2015-06-171-4/+4
| | | | | | Re-instates r239949 without accidentally flipping the sense of UseLEA. llvm-svn: 239950
* Revert "[X86] Cache variables that only depend on the subtarget"Reid Kleckner2015-06-171-4/+4
| | | | | | This reverts commit r239948, tests seem to be failing. llvm-svn: 239949
* [X86] Cache variables that only depend on the subtargetReid Kleckner2015-06-171-4/+4
| | | | | | | | | | | | | | There is a one-to-one relationship between X86Subtarget and X86FrameLowering, but every frame lowering method would previously pull the subtarget off the MachineFunction and query some subtarget properties. Over time, these locals began to grow in complexity and it became important to keep their names and meaning in sync across all of the frame lowering methods, leading to duplication. We can eliminate that duplication by computing them once in the constructor. llvm-svn: 239948
* [X86] Implement the support for shrink-wrapping.Quentin Colombet2015-05-271-2/+3
| | | | | | | | | | | | | With this patch the x86 backend is now shrink-wrapping capable and this functionality can be tested by using the -enable-shrink-wrap switch. The next step is to make more test and enable shrink-wrapping by default for x86. Related to <rdar://problem/20821487> llvm-svn: 238293
* Reapply r238011 with a fix for the trap instruction.Quentin Colombet2015-05-221-0/+188
| | | | | | | | | | | | | | | | | | | The problem was that I slipped a change required for shrink-wrapping, namely I used getFirstTerminator instead of the getLastNonDebugInstr that was here before the refactoring, whereas the surrounding code is not yet patched for that. Original message: [X86] Refactor the prologue emission to prepare for shrink-wrapping. - Add a late pass to expand pseudo instructions (tail call and EH returns). Instead of doing it in the prologue emission. - Factor some static methods in X86FrameLowering to ease code sharing. NFC. Related to <rdar://problem/20821487> llvm-svn: 238035
* Revert "[X86] Fix a variable name for r237977 so that it works with every ↵Tamas Berghammer2015-05-221-188/+0
| | | | | | | | | | | compilers." Revert "[X86] Refactor the prologue emission to prepare for shrink-wrapping." This reverts commit 6b3b93fc8b68a2c806aa992ee4bd3d7f61898d4b. This reverts commit ab0b15dff8539826283a59c2dd700a18a9680e0f. llvm-svn: 238011
* [X86] Fix a variable name for r237977 so that it works with every compilers.Quentin Colombet2015-05-221-3/+3
| | | | llvm-svn: 237980
* [X86] Refactor the prologue emission to prepare for shrink-wrapping.Quentin Colombet2015-05-221-0/+188
- Add a late pass to expand pseudo instructions (tail call and EH returns). Instead of doing it in the prologue emission. - Factor some static methods in X86FrameLowering to ease code sharing. NFC. Related to <rdar://problem/20821487> llvm-svn: 237977
OpenPOWER on IntegriCloud