summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen
Commit message (Collapse)AuthorAgeFilesLines
* [DebugInfo] Debug locations for constant SD nodesSergey Dmitrouk2015-05-131-41/+76
| | | | | | | | | | | | | | | | | | | | | Several updates for [DebugInfo] Add debug locations to constant SD nodes (r235989). Includes: * re-enabling the change (disabled recently); * missing change for FP constants; * resetting debug location of constant node if it's used more than at one place to prevent emission of wrong locations in case of coalesced constants; * a couple of additional tests. Now all look ups in CSEMap are wrapped by additional method. Comment in D9084 suggests that debug locations aren't useful for "target constants", so there might be one more change related to this API (namely, dropping debug locations for getTarget*Constant methods). Differential Revision: http://reviews.llvm.org/D9604 llvm-svn: 237237
* [Statepoints] Support for "patchable" statepoints.Sanjoy Das2015-05-122-9/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This change adds two new parameters to the statepoint intrinsic, `i64 id` and `i32 num_patch_bytes`. `id` gets propagated to the ID field in the generated StackMap section. If the `num_patch_bytes` is non-zero then the statepoint is lowered to `num_patch_bytes` bytes of nops instead of a call (the spill and reload code remains unchanged). A non-zero `num_patch_bytes` is useful in situations where a language runtime requires complete control over how a call is lowered. This change brings statepoints one step closer to patchpoints. With some additional work (that is not part of this patch) it should be possible to get rid of `TargetOpcode::STATEPOINT` altogether. PlaceSafepoints generates `statepoint` wrappers with `id` set to `0xABCDEF00` (the old default value for the ID reported in the stackmap) and `num_patch_bytes` set to `0`. This can be made more sophisticated later. Reviewers: reames, pgavlin, swaroop.sridhar, AndyAyers Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D9546 llvm-svn: 237214
* CodeGen: ignore DEBUG_VALUE nodes in KILL taggingSaleem Abdulrasool2015-05-121-0/+6
| | | | | | | DEBUG_VALUE nodes do not take part in code generation. Ignore them when performing KILL updates. Addresses PR23486. llvm-svn: 237211
* [Statepoints] Clean up statepoint argument accessors.Pat Gavlin2015-05-121-22/+12
| | | | | | Differential Revision: http://reviews.llvm.org/D9622 llvm-svn: 237191
* Convert PHI getIncomingValue() to foreach over incoming_values(). NFC.Pete Cooper2015-05-121-2/+2
| | | | | | | | We already had a method to iterate over all the incoming values of a PHI. This just changes all eligible code to use it. Ineligible code included anything which cared about the index, or was also trying to get the i'th incoming BB. llvm-svn: 237169
* [Statepoints] Split the calling convention and statepoint flags operand to ↵Pat Gavlin2015-05-121-22/+15
| | | | | | | | STATEPOINT into two separate operands. Differential Revision: http://reviews.llvm.org/D9623 llvm-svn: 237166
* Reverse ordering of base and derived pointer during safepoint lowering.Igor Laevsky2015-05-121-10/+12
| | | | | | | | According to the documentation in StackMap section for the safepoint we should have: "The first Location in each pair describes the base pointer for the object. The second is the derived pointer actually being relocated." But before this change we emitted them in reverse order - derived pointer first, base pointer second. llvm-svn: 237126
* Migrate existing backends that care about software floating pointEric Christopher2015-05-121-1/+1
| | | | | | | | | | | | | | | | | | | | to use the information in the module rather than TargetOptions. We've had and clang has used the use-soft-float attribute for some time now so have the backends set a subtarget feature based on a particular function now that subtargets are created based on functions and function attributes. For the one middle end soft float check go ahead and create an overloadable TargetLowering::useSoftFloat function that just checks the TargetSubtargetInfo in all cases. Also remove the command line option that hard codes whether or not soft-float is set by using the attribute for all of the target specific test cases - for the generic just go ahead and add the attribute in the one case that showed up. llvm-svn: 237079
* Fixing memory leakAndrew Kaylor2015-05-121-0/+2
| | | | llvm-svn: 237072
* Refactoring gc_relocate related code in CodeGenPrepare.cppSanjoy Das2015-05-111-7/+4
| | | | | | | | | | | | | | | | | | | | Summary: The original code inserted new instructions by following a Create->Remove->ReInsert flow. This patch removes the unnecessary Remove->ReInsert part by setting up the InsertPoint correctly at the very beginning. This change does not introduce any functionality change. Patch by Chen Li! Reviewers: reames, AndyAyers, sanjoy Reviewed By: sanjoy Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D9687 llvm-svn: 237070
* [WinEH] Handle nested landing pads that return directly to the parent function.Andrew Kaylor2015-05-111-10/+104
| | | | | | Differential Revision: http://reviews.llvm.org/D9684 llvm-svn: 237063
* propagate IR-level fast-math-flags to DAG nodes; 2nd try; NFCSanjay Patel2015-05-114-52/+56
| | | | | | | | | | | | | | | | | | | | | | | This is a less ambitious version of: http://reviews.llvm.org/rL236546 because that was reverted in: http://reviews.llvm.org/rL236600 because it caused memory corruption that wasn't related to FMF but was actually due to making nodes with 2 operands derive from a plain SDNode rather than a BinarySDNode. This patch adds the minimum plumbing necessary to use IR-level fast-math-flags (FMF) in the backend without actually using them for anything yet. This is a follow-on to: http://reviews.llvm.org/rL235997 ...which split the existing nsw / nuw / exact flags and FMF into their own struct. llvm-svn: 237046
* Fixing build warningsAndrew Kaylor2015-05-111-2/+2
| | | | llvm-svn: 237042
* [WinEH] Update exception numbering to give handlers their own base state.Andrew Kaylor2015-05-112-16/+87
| | | | | | Differential Revision: http://reviews.llvm.org/D9512 llvm-svn: 237014
* [RewriteStatepointsForGC] Fix a bug on creating gc_relocate for pointer to ↵Sanjoy Das2015-05-111-3/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | vector of pointers Summary: In RewriteStatepointsForGC pass, we create a gc_relocate intrinsic for each relocated pointer, and the gc_relocate has the same type with the pointer. During the creation of gc_relocate intrinsic, llvm requires to mangle its type. However, llvm does not support mangling of all possible types. RewriteStatepointsForGC will hit an assertion failure when it tries to create a gc_relocate for pointer to vector of pointers because mangling for vector of pointers is not supported. This patch changes the way RewriteStatepointsForGC pass creates gc_relocate. For each relocated pointer, we erase the type of pointers and create an unified gc_relocate of type i8 addrspace(1)*. Then a bitcast is inserted to convert the gc_relocate to the correct type. In this way, gc_relocate does not need to deal with different types of pointers and the unsupported type mangling is no longer a problem. This change would also ease further merge when LLVM erases types of pointers and introduces an unified pointer type. Some minor changes are also introduced to gc_relocate related part in InstCombineCalls, CodeGenPrepare, and Verifier accordingly. Patch by Chen Li! Reviewers: reames, AndyAyers, sanjoy Reviewed By: sanjoy Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D9592 llvm-svn: 237009
* LiveRangeCalc: Improve error messages on malformed IRMatthias Braun2015-05-111-1/+7
| | | | llvm-svn: 237008
* [SelectionDAG] Fixed constant folding issue when legalised types are smaller ↵Simon Pilgrim2015-05-101-2/+3
| | | | | | | | then the folded type. Found when testing with llvm-stress on i686 targets. llvm-svn: 236954
* Fix MergeConsecutiveStore for non-byte-sized memory accesses.James Y Knight2015-05-091-0/+4
| | | | | | | | | | | | | | | | | | | The bug showed up as a compile-time assertion failure: Assertion `NumBits >= MIN_INT_BITS && "bitwidth too small"' failed when building msan tests on x86-64. Prior to r236850, this bug was masked due to a bogus alignment check, which also accidentally rejected non-byte-sized accesses. Afterwards, an invalid ElementSizeBytes == 0 got further into the function, and triggered the assertion failure. It would probably be a good idea to allow it to handle merging stores of unusual widths as well, but for now, to un-break it, I'm just making the minimal fix. Differential Revision: http://reviews.llvm.org/D9626 llvm-svn: 236927
* MachineCSE: Add a target query for the LookAheadLimit heurisiticTom Stellard2015-05-091-2/+3
| | | | | | | | | This is used to determine whether or not to CSE physical register defs. Differential Revision: http://reviews.llvm.org/D9472 llvm-svn: 236923
* [Fast-ISel] Don't mark the first use of a remat constant as killed.Pete Cooper2015-05-091-4/+7
| | | | | | | | | | | | | | | | | | | When emitting something like 'add x, 1000' if we remat the 1000 then we should be able to mark the vreg containing 1000 as killed. Given that we go bottom up in fast-isel, a later use of 1000 will be higher up in the BB and won't kill it, or be impacted by the lower kill. However, rematerialised constant expressions aren't generated bottom up. The local value save area grows downwards. This means that if you remat 2 constant expressions which both use 1000 then the first will kill it, then the second, which is *lower* in the BB will read a killed register. This is the case in the attached test where the 2 GEPs both need to generate 'add x, 6680' for the constant offset. Note that this commit only makes kill flag generation conservative. There's nothing else obviously wrong with the local value save area growing downwards, and in fact it needs to for handling arbitrarily complex constant expressions. However, it would be nice if there was a solution which would let us generate more accurate kill flags, or just kill flags completely. llvm-svn: 236922
* ScheduleDAGInstrs: In functions with tail calls PseudoSourceValues are not ↵Arnold Schwaighofer2015-05-081-0/+8
| | | | | | | | | | | | | | | | | | | | non-aliasing distinct objects The code that builds the dependence graph assumes that two PseudoSourceValues don't alias. In a tail calling function two FixedStackObjects might refer to the same location. Worse 'immutable' fixed stack objects like function arguments are not immutable and will be clobbered. Change this so that a load from a FixedStackObject is not invariant in a tail calling function and don't return a PseudoSourceValue for an instruction in tail calling functions when building the dependence graph so that we handle function arguments conservatively. Fix for PR23459. rdar://20740035 llvm-svn: 236916
* Switch lowering: cluster adjacent fall-through cases even at -O0Hans Wennborg2015-05-081-3/+5
| | | | | | | It's cheap to do, and codegen is much faster if cases can be merged into clusters. llvm-svn: 236905
* [Fast-ISel] Clear kill flags on registers replaced by updateValueMap.Pete Cooper2015-05-081-0/+7
| | | | | | | | | | When selecting an extract instruction, we don't actually generate code but instead work out which register we are reading, and rewrite uses of the extract def to the source register. This is done via updateValueMap,. However, its possible that the source register we are rewriting *to* to also have uses. If those uses are after a kill of the value we are rewriting *from* then we have uses after a kill and the verifier fails. This code checks for the case where the to register is also used, and if so it clears all kill on the from register. This is conservative, but better that always clearing kills on the from register. llvm-svn: 236897
* Extend the statepoint intrinsic to allow statepoints to be marked as ↵Pat Gavlin2015-05-082-11/+93
| | | | | | | | | | | | | | | | | | | | | | transitions from GC-aware code to code that is not GC-aware. This changes the shape of the statepoint intrinsic from: @llvm.experimental.gc.statepoint(anyptr target, i32 # call args, i32 unused, ...call args, i32 # deopt args, ...deopt args, ...gc args) to: @llvm.experimental.gc.statepoint(anyptr target, i32 # call args, i32 flags, ...call args, i32 # transition args, ...transition args, i32 # deopt args, ...deopt args, ...gc args) This extension offers the backend the opportunity to insert (somewhat) arbitrary code to manage the transition from GC-aware code to code that is not GC-aware and back. In order to support the injection of transition code, this extension wraps the STATEPOINT ISD node generated by the usual lowering lowering with two additional nodes: GC_TRANSITION_START and GC_TRANSITION_END. The transition arguments that were passed passed to the intrinsic (if any) are lowered and provided as operands to these nodes and may be used by the backend during code generation. Eventually, the lowering of the GC_TRANSITION_{START,END} nodes should be informed by the GC strategy in use for the function containing the intrinsic call; for now, these nodes are instead replaced with no-ops. Differential Revision: http://reviews.llvm.org/D9501 llvm-svn: 236888
* Clear kill flags on all used registers when sinking instructions.Pete Cooper2015-05-081-1/+7
| | | | | | | | | | | | | The test here was sinking the AND here to a lower BB: %vreg7<def> = ANDWri %vreg8, 0; GPR32common:%vreg7,%vreg8 TBNZW %vreg8<kill>, 0, <BB#1>; GPR32common:%vreg8 which meant that vreg8 was read after it was killed. This commit changes the code from clearing kill flags on the AND to clearing flags on all registers used by the AND. llvm-svn: 236886
* 80 cols fix since i'm looking at this function anyway. NFCPete Cooper2015-05-081-1/+2
| | | | llvm-svn: 236885
* Fix alignment checks in MergeConsecutiveStores.James Y Knight2015-05-081-36/+52
| | | | | | | | | | | | | | | 1) check whether the alignment of the memory is sufficient for the *merged* store or load to be efficient. Not doing so can result in some ridiculously poor code generation, if merging creates a vector operation which must be aligned but isn't. 2) DON'T check that the alignment of each load/store is equal. If you're merging 2 4-byte stores, the first *might* have 8-byte alignment, but the second certainly will have 4-byte alignment. We do want to allow those to be merged. llvm-svn: 236850
* Fix coding standart based on post submit comments.Igor Laevsky2015-05-081-4/+4
| | | | | | Differential Revision: http://reviews.llvm.org/D7760 llvm-svn: 236849
* Clear kill flags in tail duplication.Pete Cooper2015-05-071-0/+3
| | | | | | | | | | | | | | If we duplicate an instruction then we must also clear kill flags on any uses we rewrite. Otherwise we might be killing a register which was used in other BBs. For example, here the entry BB ended up with these instructions, the ADD having been tail duplicated. %vreg24<def> = t2ADDri %vreg10<kill>, 1, pred:14, pred:%noreg, opt:%noreg; GPRnopc:%vreg24 rGPR:%vreg10 %vreg22<def> = COPY %vreg10; GPR:%vreg22 rGPR:%vreg10 The copy here is inserted after the add and so needs vreg10 to be live. llvm-svn: 236782
* Switch lowering: handle zero-weight branch probabilitiesHans Wennborg2015-05-071-16/+7
| | | | | | | | | | After r236617, branch probabilities are no longer guaranteed to be >= 1. This patch makes the swich lowering code handle that correctly, without bumping the branch weights by 1 which might cause overflow and skews the probabilities. Covered by @zero_weight_tree in test/CodeGen/X86/switch.ll. llvm-svn: 236739
* Handle dead defs in the if converter.Pete Cooper2015-05-062-5/+16
| | | | | | | | | | | | | | | | | | | | | | | | We had code such as this: r2 = ... t2Bcc label1: ldr ... r2 label2; return r2<dead, def> The if converter was transforming this to r2<def> = ... return [pred] r2<dead,def> ldr <r2, kill> return which fails the machine verifier because the ldr now reads from a dead def. The fix here detects dead defs in stepForward and passes them back to the caller in the clobbers list. The caller then clears the dead flag from the def is the value is live. llvm-svn: 236660
* [RegisterCoalescer] Make sure each live-range has only one component, asQuentin Colombet2015-05-061-4/+30
| | | | | | | | | | | | | demanded by the machine verifier. After shrinking a live-range to its uses, it is possible to create several smaller live-ranges. When this happens, shrinkToUses returns true and we need to split the different components into their own live-ranges. The problem does not reproduce on any in-tree target but Jonas Paulsson <jonas.paulsson@ericsson.com>, who reported the problem, checked that this patch fixes the issue. llvm-svn: 236658
* Fix incorrect kill flags in fastisel.Pete Cooper2015-05-061-2/+6
| | | | | | | | If called twice in the same BB on the same constant, FastISel::fastEmit_ri_ was marking the materialized vreg as killed on each use, instead of only the last use. Change this to only mark the last use as killed by making earlier uses check if the vreg is already used elsewhere. llvm-svn: 236650
* MC: Skip names of temporary symbols in object streamerDuncan P. N. Exon Smith2015-05-061-0/+3
| | | | | | | | | | | | | | Don't create names for temporary symbols when using an object streamer. The names never make it to the output anyway. From the starting point of r236629, my heap profile says this drops peak memory usage from 1100 MB to 1058 MB for CodeGen of `verify-uselistorder`, a savings of almost 4% on peak memory, and removes `StringMap<bool, BumpPtrAllocator...>` from the profile entirely. (I'm looking at `llc` memory usage on `verify-uselistorder.lto.opt.bc`; see r236629 for details.) llvm-svn: 236642
* CodeGen: move over-zealous assert into actual if statement.Tim Northover2015-05-061-3/+2
| | | | | | | | | | | | | | | It's quite possible to encounter an insertvalue instruction that's more deeply nested than the value we're looking for, but when that happens we really mustn't compare beyond the end of the index array. Since I couldn't see any guarantees about what comparisons std::equal makes, we probably need to directly check the size beforehand. In practice, I suspect most std::equal implementations would probably bail early, which would be OK. But just in case... rdar://20834485 llvm-svn: 236635
* DwarfDebug: Emit number of bytes in .debug_loc entry directlyDuncan P. N. Exon Smith2015-05-061-6/+3
| | | | | | | | | | | | | | | | | | | | | | Emit the number of bytes in a `.debug_loc` entry directly. The old code created temp labels (expensive), emitted the difference between them, and then emitted one on each side of the relevant bytes. (I'm looking at `llc` memory usage on `verify-uselistorder.lto.opt.bc` (the optimized version of ld64's `-save-temps` when linking the `verify-uselistorder` executable in an LTO bootstrap). I've hacked `MCContext::Allocate()` to just call `malloc()` instead of using the `BumpPtrAllocator` so that the heap profile is easier to read. As far as peak memory is concerned, `MCContext::Allocate()` is equivalent to a leak, since it only gets freed at process teardown. In my heap profile, this patch drops memory usage of `DwarfDebug::emitDebugLoc()` from 132.56 MB (11.4%) down to 29.86 MB (2.7%) at peak memory. Some of that must be noise from `SmallVector` (or other) allocations -- peak memory only dropped from 1160 MB down to 1100 MB -- but this nevertheless shaves 5% off the top.) llvm-svn: 236629
* [WinEH] Improve fatal error message about failed demotionReid Kleckner2015-05-061-1/+6
| | | | llvm-svn: 236626
* [SelectionDAG] Delete SelectionDAGBuilder::removeValue. NFC.Sanjoy Das2015-05-061-6/+0
| | | | | | SelectionDAGBuilder::removeValue is dead now, after rL236563. llvm-svn: 236618
* Allow 0-weight branches in BranchProbabilityInfo.Diego Novillo2015-05-061-1/+5
| | | | | | | | | | | | | | | | | | | | | | | Summary: When computing branch weights in BPI, we used to disallow branches with weight 0. This is a minor nuisance, because a branch with weight 0 is different to "don't have information". In the context of instrumentation, it may mean "never executed", in the context of sampling, it means "never or seldom executed". In allowing 0 weight branches, I ran into issues with the switch expansion code in selection DAG. It is currently hardwired to not handle branches with weight 0. To maintain the current behaviour, I changed it to use 1 when it finds 0, but perhaps the algorithm needs changes to tolerate branches with weight zero. Reviewers: hansw Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D9533 llvm-svn: 236617
* Add ChangeTo* to MachineOperand for symbolsMatt Arsenault2015-05-061-0/+22
| | | | llvm-svn: 236612
* Reformat.NAKAMURA Takumi2015-05-062-6/+5
| | | | llvm-svn: 236601
* Revert r236546, "propagate IR-level fast-math-flags to DAG nodes (NFC)"NAKAMURA Takumi2015-05-064-65/+60
| | | | | | It caused undefined behavior. llvm-svn: 236600
* SelectionDAG: Handle out-of-bounds index in extract vector elementPawel Bylica2015-05-061-0/+4
| | | | | | | | | | | | | | | | | | Summary: This patch correctly handles undef case of EXTRACT_VECTOR_ELT node where the element index is constant and not less than vector size. Test Plan: CodeGen for X86 test included. Also one incorrect regression test fixed. Reviewers: qcolombet, chandlerc, hfinkel Reviewed By: hfinkel Subscribers: hfinkel, llvm-commits Differential Revision: http://reviews.llvm.org/D9250 llvm-svn: 236584
* [Statepoint] Clean up StatepointLowering: symbolic constants.Sanjoy Das2015-05-061-2/+3
| | | | | | | | For accessors in the `Statepoint` class, use symbolic constants for offsets into the argument vector instead of literals. This makes the code intent clearer and simpler to change. llvm-svn: 236566
* [Statepoint] Clean up Statepoint.h: accessor names.Sanjoy Das2015-05-062-16/+16
| | | | | | Use getFoo() as accessors consistently and some other naming changes. llvm-svn: 236564
* [StatepointLowering] Don't create temporary instructions. NFCI.Sanjoy Das2015-05-061-73/+69
| | | | | | | | | | | | | | Summary: Instead of creating a temporary call instruction and lowering that, use SelectionDAGBuilder::lowerCallOperands. Reviewers: reames Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D9480 llvm-svn: 236563
* [WinEH] Reset WinEHPrepare::SEHExceptionCodeSlot when we're done.Ahmed Bougacha2015-05-061-0/+1
| | | | | | | This caused a use-after-free on test/CodeGen/X86/win32-eh.ll No functional change intended. llvm-svn: 236561
* [SelectionDAG] Make an argument optional in RFV::getCopyToRegs. NFC.Sanjoy Das2015-05-051-5/+6
| | | | | | | | | | | | | | | Summary: We default the value argument to nullptr. The only use of the value is in diagnosePossiblyInvalidConstraint and that seems to be resilient to it being nullptr. Reviewers: atrick, reames Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D9479 llvm-svn: 236555
* [SelectionDAG] Move RegsForValue into SelectionDAGBuilder.h. NFC.Sanjoy Das2015-05-052-85/+90
| | | | | | | | | | | | | | | Summary: The exported class will be used in later change, in StatepointLowering.cpp. It is still internal to SelectionDAG (not exported via include/). Reviewers: reames, atrick Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D9478 llvm-svn: 236554
* [SelectionDAG] Pass explicit type to lowerCallOperands. NFC.Sanjoy Das2015-05-052-5/+6
| | | | | | | | | | | | | | Summary: Currently this does not change anything, but change will be used in a later change to StatepointLowering.cpp Reviewers: reames, atrick Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D9477 llvm-svn: 236553
OpenPOWER on IntegriCloud