summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen
Commit message (Collapse)AuthorAgeFilesLines
* CodeGen: Use range-based for loops, NFCDuncan P. N. Exon Smith2015-01-081-5/+5
| | | | | | Patch by Ramkumar Ramachandra! llvm-svn: 225466
* Masked Load/Store - fixed a bug in type legalization.Elena Demikhovsky2015-01-083-3/+107
| | | | llvm-svn: 225441
* Fix include ordering, NFC.Michael Kuperstein2015-01-081-1/+1
| | | | llvm-svn: 225439
* Move SPAdj logic from PEI into the targets (NFC)Michael Kuperstein2015-01-082-11/+34
| | | | | | | | | | | | PEI tries to keep track of how much starting or ending a call sequence adjusts the stack pointer by, so that it can resolve frame-index references. Currently, it takes a very simplistic view of how SP adjustments are done - both FrameStartOpcode and FrameDestroyOpcode adjust it exactly by the amount written in its first argument. This view is in fact incorrect for some targets (e.g. due to stack re-alignment, or because it may want to adjust the stack pointer in multiple steps). However, that doesn't cause breakage, because most targets (the only in-tree exception appears to be 32-bit ARM) rely on being able to simplify the call frame pseudo-instructions earlier, so this code is never hit. Moving the computation into TargetInstrInfo allows targets to override the way the adjustment is computed if they need to have a non-zero SPAdj. Differential Revision: http://reviews.llvm.org/D6863 llvm-svn: 225437
* [RegAllocGreedy] Introduce a late pass to repair broken hints.Quentin Colombet2015-01-083-2/+212
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A broken hint is a copy where both ends are assigned different colors. When a variable gets evicted in the neighborhood of such copies, it is likely we can reconcile some of them. ** Context ** Copies are inserted during the register allocation via splitting. These split points are required to relax the constraints on the allocation problem. When such a point is inserted, both ends of the copy would not share the same color with respect to the current allocation problem. When variables get evicted, the allocation problem becomes different and some split point may not be required anymore. However, the related variables may already have been colored. This usually shows up in the assembly with pattern like this: def A ... save A to B def A use A restore A from B ... use B Whereas we could simply have done: def B ... def A use A ... use B ** Proposed Solution ** A variable having a broken hint is marked for late recoloring if and only if selecting a register for it evict another variable. Indeed, if no eviction happens this is pointless to look for recoloring opportunities as it means the situation was the same as the initial allocation problem where we had to break the hint. Finally, when everything has been allocated, we look for recoloring opportunities for all the identified candidates. The recoloring is performed very late to rely on accurate copy cost (all involved variables are allocated). The recoloring is simple unlike the last change recoloring. It propagates the color of the broken hint to all its copy-related variables. If the color is available for them, the recoloring uses it, otherwise it gives up on that hint even if a more complex coloring would have worked. The recoloring happens only if it is profitable. The profitability is evaluated using the expected frequency of the copies of the currently recolored variable with a) its current color and b) with the target color. If a) is greater or equal than b), then it is profitable and the recoloring happen. ** Example ** Consider the following example: BB1: a = b = BB2: ... = b = a Let us assume b gets split: BB1: a = b = BB2: c = b ... d = c = d = a Because of how the allocation work, b, c, and d may be assigned different colors. Now, if a gets evicted to make room for c, assuming b and d were assigned to something different than a. We end up with: BB1: a = st a, SpillSlot b = BB2: c = b ... d = c = d e = ld SpillSlot = e This is likely that we can assign the same register for b, c, and d, getting rid of 2 copies. ** Performances ** Both ARM64 and x86_64 show performance improvements of up to 3% for the llvm-testsuite + externals with Os and O3. There are a few regressions too that comes from the (in)accuracy of the block frequency estimate. <rdar://problem/18312047> llvm-svn: 225422
* [SelectionDAG] Allow targets to specify legality of extloads' resultAhmed Bougacha2015-01-085-33/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | type (in addition to the memory type). The *LoadExt* legalization handling used to only have one type, the memory type. This forced users to assume that as long as the extload for the memory type was declared legal, and the result type was legal, the whole extload was legal. However, this isn't always the case. For instance, on X86, with AVX, this is legal: v4i32 load, zext from v4i8 but this isn't: v4i64 load, zext from v4i8 Whereas v4i64 is (arguably) legal, even without AVX2. Note that the same thing was done a while ago for truncstores (r46140), but I assume no one needed it yet for extloads, so here we go. Calls to getLoadExtAction were changed to add the value type, found manually in the surrounding code. Calls to setLoadExtAction were mechanically changed, by wrapping the call in a loop, to match previous behavior. The loop iterates over the MVT subrange corresponding to the memory type (FP vectors, etc...). I also pulled neighboring setTruncStoreActions into some of the loops; those shouldn't make a difference, as the additional types are illegal. (e.g., i128->i1 truncstores on PPC.) No functional change intended. Differential Revision: http://reviews.llvm.org/D6532 llvm-svn: 225421
* RegisterCoalescer: Do not remove IMPLICIT_DEFS if they are required for ↵Matthias Braun2015-01-081-1/+7
| | | | | | | | | | | subranges. The register coalescer used to remove implicit_defs when they are covered by the main range anyway. With subreg liveness tracking we can't do that anymore in places where the IMPLICIT_DEF is required as begin of a subregister liverange. llvm-svn: 225416
* RegisterCoalescer: Fix valuesIdentical() in some subrange merge cases.Matthias Braun2015-01-071-90/+81
| | | | | | | | | | | | | I got confused and assumed SrcIdx/DstIdx of the CoalescerPair is a subregister index in SrcReg/DstReg, but they are actually subregister indices of the coalesced register that get you back to SrcReg/DstReg when applied. Fixed the bug, improved comments and simplified code accordingly. Testcase by Tom Stellard! llvm-svn: 225415
* LiveInterval: Implement feedback by Quentin Colombet.Matthias Braun2015-01-071-25/+32
| | | | llvm-svn: 225413
* Update a comment.Adrian Prantl2015-01-071-0/+2
| | | | llvm-svn: 225399
* [CodeGen] Use MVT iterator_ranges in legality loops. NFC intended.Ahmed Bougacha2015-01-071-19/+14
| | | | | | | | A few loops do trickier things than just iterating on an MVT subset, so I'll leave them be for now. Follow-up of r225387. llvm-svn: 225392
* More FMA folding opportunities.Olivier Sallenave2015-01-071-1/+133
| | | | llvm-svn: 225380
* Debug info: Allow aggregate types to be described by constants.Adrian Prantl2015-01-071-2/+5
| | | | llvm-svn: 225378
* Test commitOlivier Sallenave2015-01-071-0/+1
| | | | llvm-svn: 225368
* Add a missing file from 225365Philip Reames2015-01-071-0/+54
| | | | llvm-svn: 225366
* Introduce an example statepoint GC strategyPhilip Reames2015-01-074-0/+50
| | | | | | | | | | | | | | This change includes the most basic possible GCStrategy for a GC which is using the statepoint lowering code. At the moment, this GCStrategy doesn't really do much - aside from actually generate correct stackmaps that is - but I went ahead and added a few extra correctness checks as proof of concept. It's mostly here to provide documentation on how to do one, and to provide a point for various optimization legality hooks I'd like to add going forward. (For context, see the TODOs in InstCombine around gc.relocate.) Most of the validation logic added here as proof of concept will soon move in to the Verifier. That move is dependent on http://reviews.llvm.org/D6811 There was discussion in the review thread about addrspace(1) being reserved for something. I'm going to follow up on a seperate llvmdev thread. If needed, I'll update all the code at once. Note that I am deliberately not making a GCStrategy required to use gc.statepoints with this change. I want to give folks out of tree - including myself - a chance to migrate. In a week or two, I'll make having a GCStrategy be required for gc.statepoints. To this end, I added the gc tag to one of the test cases but not others. Differential Revision: http://reviews.llvm.org/D6808 llvm-svn: 225365
* New method SDep::isNormalMemoryOrBarrier() in ScheduleDAGInstrs.cpp.Jonas Paulsson2015-01-071-5/+6
| | | | | | | | | | | | | | | | | | | | Used to iterate over previously added memory dependencies in adjustChainDeps() and iterateChainSucc(). SDep::isCtrl() was previously used in these places, that also gave anti and output edges. The code may be worse if these are followed, because MisNeedChainEdge() will conservatively return true since a non-memory instruction has no memory operands, and a false chain dep will be added. It is also unnecessary since all memory accesses of interest will be reached by memory dependencies, and there is a budget limit for the number of edges traversed. This problem was found on an out-of-tree target with enabled alias analysis. No test case for an in-tree target has been found. Reviewed by Hal Finkel. llvm-svn: 225351
* Fix typos in comment and option help texts.Jonas Paulsson2015-01-071-3/+3
| | | | | | For -enable-aa-sched-mi and -use-tbaa-in-sched-mi. llvm-svn: 225350
* Revert r224935 "Refactor duplicated code. No intended functionality change."Lang Hames2015-01-061-0/+54
| | | | | | | | This is affecting the behavior of some ObjC++ / AArch64 test cases on Darwin. Reverting to get the bots green while I track down the source of the changed behavior. llvm-svn: 225311
* SelectionDAGBuilder: move constant initialization out of loopMehdi Amini2015-01-061-15/+19
| | | | | | | | | | No semantic change intended. Reviewers: resistor Differential Revision: http://reviews.llvm.org/D6834 llvm-svn: 225278
* [CodeGenPrepare] Improved logic to speculate calls to cttz/ctlz.Andrea Di Biagio2015-01-061-6/+35
| | | | | | | | | | | | | | | | | | | | | This patch improves the logic added at revision 224899 (see review D6728) that teaches the backend when it is profitable to speculate calls to cttz/ctlz. The original algorithm conservatively avoided speculating more than one instruction from a basic block in a control flow grap modelling an if-statement. In particular, the only allowed instruction (excluding the terminator) was a call to cttz/ctlz. However, there are cases where we could be less conservative and still be able to speculate a call to cttz/ctlz. With this patch, CodeGenPrepare now tries to speculate a cttz/ctlz if the result is zero extended/truncated in the same basic block, and the zext/trunc instruction is "free" for the target. Added new test cases to CodeGen/X86/cttz-ctlz.ll Differential Revision: http://reviews.llvm.org/D6853 llvm-svn: 225274
* Make DIE.h a public CodeGen header.Frederic Riss2015-01-059-595/+8
| | | | | | | | | | | | | | dsymutil would like to use all the AsmPrinter/MCStreamer infrastructure to stream out the DWARF. In order to do so, it will reuse the DIE object and so this header needs to be public. The interface exposed here has some corners that cannot be used without a DwarfDebug object, but clients that want to stream Dwarf can just avoid these. Differential Revision: http://reviews.llvm.org/D6695 llvm-svn: 225208
* Replace several 'assert(false' with 'llvm_unreachable' or fold a condition ↵Craig Topper2015-01-053-4/+5
| | | | | | into the assert. llvm-svn: 225160
* [PowerPC/BlockPlacement] Allow target to provide a per-loop alignment preferenceHal Finkel2015-01-031-3/+4
| | | | | | | | | | | | | | | | | The existing code provided for specifying a global loop alignment preference. However, the preferred loop alignment might depend on the loop itself. For recent POWER cores, loops between 5 and 8 instructions should have 32-byte alignment (while the others are better with 16-byte alignment) so that the entire loop will fit in one i-cache line. To support this, getPrefLoopAlignment has been made virtual, and can be provided with an optional MachineLoop* so the target can inspect the loop before answering the query. The default behavior, as before, is to return the value set with setPrefLoopAlignment. MachineBlockPlacement now queries the target for each loop instead of only once per function. There should be no functional change for other targets. llvm-svn: 225117
* Revert "merge consecutive stores of extracted vector elements"Alexey Samsonov2014-12-311-75/+4
| | | | | | | This reverts commit r224611. This change causes crashes in X86 DAG->DAG Instruction Selection. llvm-svn: 225031
* DebugInfo: Omit is_stmt from line table entries on the same line.David Blaikie2014-12-301-1/+3
| | | | | | | | | | | | | | | | | | | | | | GCC does this for non-zero discriminators and since GCC doesn't produce column info, that was the only place it comes up there. For LLVM, since we can emit discriminators and/or column info, it makes more sense to invert the condition and just test for changes in line number. This should resolve at least some of the GDB 7.5 test suite failures created by recent Clang changes that increase the location fidelity (which, since Clang defaults to including column info on Linux by default created a bunch of cases that confused GDB). In theory we could do this better/differently by grouping actual source statements together in a similar manner to the way lexical scopes are handled but given that GDB isn't really in a position to consume that (& users are probably somewhat used to different lines being different 'statements') this seems the safest and cheapest change. (I'm concerned that doing this 'right' would bloat the debugloc data even further - something Duncan's working hard to address) llvm-svn: 225011
* x86_64: Fix calls to __morestack under the large code model.Peter Collingbourne2014-12-302-1/+18
| | | | | | | | | | | | | | | | | | | | Under the large code model, we cannot assume that __morestack lives within 2^31 bytes of the call site, so we cannot use pc-relative addressing. We cannot perform the call via a temporary register, as the rax register may be used to store the static chain, and all other suitable registers may be either callee-save or used for parameter passing. We cannot use the stack at this point either because __morestack manipulates the stack directly. To avoid these issues, perform an indirect call via a read-only memory location containing the address. This solution is not perfect, as it assumes that the .rodata section is laid out within 2^31 bytes of each function body, but this seems to be sufficient for JIT. Differential Revision: http://reviews.llvm.org/D6787 llvm-svn: 225003
* [COFF] Don't try to add quotes to already quoted linker directivesMichael Kuperstein2014-12-301-1/+1
| | | | | | | | | | If a linker directive is already quoted, don't try to quote it again, otherwise it creates a mess. This pops up in places like: #pragma comment(linker,"\"/foo bar'\"") Differential Revision: http://reviews.llvm.org/D6792 llvm-svn: 224998
* Refactor duplicated code.Rafael Espindola2014-12-291-54/+0
| | | | | | No intended functionality change. llvm-svn: 224935
* [CodeGenPrepare] Teach when it is profitable to speculate calls to ↵Andrea Di Biagio2014-12-281-0/+141
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | @llvm.cttz/ctlz. If the control flow is modelling an if-statement where the only instruction in the 'then' basic block (excluding the terminator) is a call to cttz/ctlz, CodeGenPrepare can try to speculate the cttz/ctlz call and simplify the control flow graph. Example: \code entry: %cmp = icmp eq i64 %val, 0 br i1 %cmp, label %end.bb, label %then.bb then.bb: %c = tail call i64 @llvm.cttz.i64(i64 %val, i1 true) br label %end.bb end.bb: %cond = phi i64 [ %c, %then.bb ], [ 64, %entry] \code In this example, basic block %then.bb is taken if value %val is not zero. Also, the phi node in %end.bb would propagate the size-of in bits of %val only if %val is equal to zero. With this patch, CodeGenPrepare will try to hoist the call to cttz from %then.bb into basic block %entry only if cttz is cheap to speculate for the target. Added two new hooks in TargetLowering.h to let targets customize the behavior (i.e. decide whether it is cheap or not to speculate calls to cttz/ctlz). The two new methods are 'isCheapToSpeculateCtlz' and 'isCheapToSpeculateCttz'. By default, both methods return 'false'. On X86, method 'isCheapToSpeculateCtlz' returns true only if the target has LZCNT. Method 'isCheapToSpeculateCttz' only returns true if the target has BMI. Differential Revision: http://reviews.llvm.org/D6728 llvm-svn: 224899
* Scalarizer for masked load and store intrinsics.Elena Demikhovsky2014-12-281-40/+274
| | | | | | | | Masked vector intrinsics are a part of common LLVM IR, but they are really supported on AVX2 and AVX-512 targets. I added a code that translates masked intrinsic for all other targets. The masked vector intrinsic is converted to a chain of scalar operations inside conditional basic blocks. http://reviews.llvm.org/D6436 llvm-svn: 224897
* Band-aid fix for PR22032: don't emit DWARF debug info if AddressSanitizer is ↵Timur Iskhodzhanov2014-12-261-3/+16
| | | | | | enabled on Windows llvm-svn: 224860
* Silence GCC's -Wparentheses warningDavid Majnemer2014-12-251-1/+1
| | | | | | No functionality change intended. llvm-svn: 224833
* Masked Load/Store - Changed the order of parameters in intrinsics.Elena Demikhovsky2014-12-251-5/+7
| | | | | | | No functional changes. The documentation is coming. llvm-svn: 224829
* CodeGen: Error on redefinitions instead of assertingDavid Majnemer2014-12-241-5/+11
| | | | | | | It's possible to have a prior definition of a symbol in module asm. Raise an error instead of crashing. llvm-svn: 224828
* CodeGen: Allow aliases to be overridden by variablesDavid Majnemer2014-12-241-0/+2
| | | | llvm-svn: 224827
* MC: Label definitions are permitted after .set directivesDavid Majnemer2014-12-241-0/+2
| | | | | | | | | .set directives may be overridden by other .set directives as well as label definitions. This fixes PR22019. llvm-svn: 224811
* LiveInterval: Remove accidentally committed debug code.Matthias Braun2014-12-241-10/+0
| | | | llvm-svn: 224807
* LiveInterval: Introduce createMainRangeFromSubranges().Matthias Braun2014-12-242-7/+226
| | | | | | | | | | | | | | | | | | | | This function constructs the main liverange by merging all subranges if subregister liveness tracking is available. This should be slightly faster to compute instead of performing the liveness calculation again for the main range. More importantly it avoids cases where the main liverange would cover positions where no subrange was live. These cases happened for partial definitions where the actual defined part was dead and only the undefined parts used later. The register coalescing requires that every part covered by the main live range has at least one subrange live. I also expect this function to become usefull later for places where the subranges are modified in a way that it is hard to correctly fix the main liverange in the machine scheduler, we can simply reconstruct it from subranges then. llvm-svn: 224806
* RegisterCoalescer: With subrange liveness there may be no RedefVNI for ↵Matthias Braun2014-12-241-3/+6
| | | | | | unused lanes. llvm-svn: 224805
* LiveRangeEdit: Check for completely empy subranges after removing ValNos.Matthias Braun2014-12-241-0/+1
| | | | | | | Completely empty subranges are not allowed and must be removed when subreg liveness is enabled. llvm-svn: 224804
* LiveIntervalAnalysis: Fix performance bug that I introduced in r224663.Matthias Braun2014-12-241-2/+2
| | | | | | | | Without a reference the code did not remember when moving the iterators of the subranges/registerunit ranges forward and instead would scan from the beginning again at the next position. llvm-svn: 224803
* Debug Info: In symmetry to DW_TAG_pointer_type, do not emit the byte sizeAdrian Prantl2014-12-241-1/+2
| | | | | | | of a DW_TAG_ptr_to_member_type. This restores the behavior from before r224780-r224781. llvm-svn: 224799
* Always assert in DAGCombine and not only when -debug is enabledMehdi Amini2014-12-231-5/+6
| | | | | | | | | Right now in DAG Combine check the validity of the returned type only when -debug is given on the command line. However usually the test cases in the validation does not use -debug. An Assert build should always check this. llvm-svn: 224779
* [DagCombine] Improve DAGCombiner BUILD_VECTOR when it has two sources of ↵Michael Kuperstein2014-12-231-12/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | elements This partially fixes PR21943. For AVX, we go from: vmovq (%rsi), %xmm0 vmovq (%rdi), %xmm1 vpermilps $-27, %xmm1, %xmm2 ## xmm2 = xmm1[1,1,2,3] vinsertps $16, %xmm2, %xmm1, %xmm1 ## xmm1 = xmm1[0],xmm2[0],xmm1[2,3] vinsertps $32, %xmm0, %xmm1, %xmm1 ## xmm1 = xmm1[0,1],xmm0[0],xmm1[3] vpermilps $-27, %xmm0, %xmm0 ## xmm0 = xmm0[1,1,2,3] vinsertps $48, %xmm0, %xmm1, %xmm0 ## xmm0 = xmm1[0,1,2],xmm0[0] To the expected: vmovq (%rdi), %xmm0 vmovhpd (%rsi), %xmm0, %xmm0 retq Fixing this for AVX2 is still open. Differential Revision: http://reviews.llvm.org/D6749 llvm-svn: 224759
* Make musttail more robust for vector types on x86Reid Kleckner2014-12-221-0/+56
| | | | | | | | | | | | | | | | Previously I tried to plug musttail into the existing vararg lowering code. That turned out to be a mistake, because non-vararg calls use significantly different register lowering, even on x86. For example, AVX vectors are usually passed in registers to normal functions and memory to vararg functions. Now musttail uses a completely separate lowering. Hopefully this can be used as the basis for non-x86 perfect forwarding. Reviewers: majnemer Differential Revision: http://reviews.llvm.org/D6156 llvm-svn: 224745
* [CodeGenPrepare] Handle properly the promotion of operands when this does notQuentin Colombet2014-12-221-3/+7
| | | | | | | | | generate instructions. Fixes PR21978. Related to <rdar://problem/18310086> llvm-svn: 224717
* The leak detector is dead, long live asan and valgrind.Rafael Espindola2014-12-221-5/+0
| | | | | | | In resent times asan and valgrind have found way more memory management bugs in llvm than the special purpose leak detector. llvm-svn: 224703
* CodeGen: minor style tweaks to SSPSaleem Abdulrasool2014-12-211-13/+15
| | | | | | Clean up some style related things in the StackProtector CodeGen. NFC. llvm-svn: 224693
* Enable (sext x) == C --> x == (trunc C) combineMatt Arsenault2014-12-211-9/+26
| | | | | | | | | Extend the existing code which handles this for zext. This makes this more useful for targets with ZeroOrNegativeOne BooleanContent and obsoletes a custom combine SI uses for i1 setcc (sext(i1), 0, setne) since the constant will now be shrunk to i1. llvm-svn: 224691
OpenPOWER on IntegriCloud