summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen
Commit message (Collapse)AuthorAgeFilesLines
...
* [legalize-types] Remove stale entries from SoftenedFloats.Florian Hahn2017-03-041-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | Summary: When replacing a SDValue, we should remove the replaced value from SoftenedFloats (and possibly the other maps as well?). When we revisit a Node because it needs analyzing again, we have to remove all result values from SoftenedFloats (and possibly other maps?). This fixes the fp128 test failures with expensive checks for X86. I think we probably should also remove the values from the other maps (PromotedIntegers and so on), let me know what you think. Reviewers: baldrick, bogner, davidxl, ab, arsenm, pirama, chh, RKSimon Reviewed By: chh Subscribers: danalbert, wdng, srhines, hfinkel, sepavloff, llvm-commits Differential Revision: https://reviews.llvm.org/D29265 llvm-svn: 296964
* [MISched] Remove unused arguments. NFC.Eli Friedman2017-03-041-4/+2
| | | | llvm-svn: 296934
* RegAllocGreedy: Follow-up to r296722Matthias Braun2017-03-031-1/+5
| | | | | | | | | We can now end up in situations where we initiate LiveIntervalUnion queries with different SubRanges against the same register unit, so the assert() no longer holds in all cases. Just recalculate now when we know the cache is out of date. llvm-svn: 296928
* GlobalISel: constrain G_INSERT to inserting just one value per instruction.Tim Northover2017-03-031-0/+9
| | | | | | | It's much easier to reason about single-value inserts and no-one was actually using the variadic variants before. llvm-svn: 296923
* GlobalISel: add merge/unmerge nodes for legalization.Tim Northover2017-03-034-19/+80
| | | | | | | | | | | | | | These are simplified variants of the current G_SEQUENCE and G_EXTRACT, which assume the individual parts will be contiguous, homogeneous, and occupy the entirity of the larger register. This makes reasoning about them much easer since you only have to look at the first register being merged and the result to know what the instruction is doing. I intend to gradually replace all uses of the more complicated sequence/extract with these (or single-element insert/extracts), and then remove the older variants. For now we start with legalization. llvm-svn: 296921
* RegisterCoalescer: Simplify subrange splitting code; NFCMatthias Braun2017-03-033-94/+51
| | | | | | - Use slightly better variable names / compute in a more direct way. llvm-svn: 296905
* Use APInt::setBits instead of OR'ing in a separate APInt::getBitsSet callSimon Pilgrim2017-03-031-1/+1
| | | | llvm-svn: 296886
* Use APInt::getOneBitSet instead of APInt::getBitsSet for sign bit mask creationSimon Pilgrim2017-03-031-1/+1
| | | | | | Avoids all the unnecessary extra bitrange creation/shift stages. llvm-svn: 296879
* Use APInt::getOneBitSet instead of APInt::getBitsSet for sign bit mask creationSimon Pilgrim2017-03-031-1/+1
| | | | | | Avoids all the unnecessary extra bitrange creation/shift stages. llvm-svn: 296871
* Fix Wdocumentation warningSimon Pilgrim2017-03-031-3/+3
| | | | llvm-svn: 296866
* [SDAG] Revert r296476 (and r296486, r296668, r296690).Chandler Carruth2017-03-033-380/+371
| | | | | | | | | | This patch causes compile times for some patterns to explode. I have a (large, unreduced) test case that slows down by more than 20x and several test cases slow down by 2x. I'm sending some of the test cases directly to Nirav and following up with more details in the review log, but this should unblock anyone else hitting this. llvm-svn: 296862
* LiveDebugValues: Assume calls never clobber SP.Adrian Prantl2017-03-031-1/+5
| | | | | | | | | | | | | | A call should never modify the stack pointer, but some backends are not so sure about this and never list SP in the regmask. For the purposes of LiveDebugValues we assume a call never clobbers SP. We already have a similar workaround in DbgValueHistoryCalculator (which we hopefully can retire soon). This fixes the availabilty of local ASANified variables on AArch64. rdar://problem/27757381 llvm-svn: 296847
* CodeGen: BlockPlacement: Precompute layout for chains of triangles.Kyle Butt2017-03-031-0/+134
| | | | | | | | | | | | | | | | | | | | | | | | | | For chains of triangles with small join blocks that can be tail duplicated, a simple calculation of probabilities is insufficient. Tail duplication can be profitable in 3 different ways for these cases: 1) The post-dominators marked 50% are actually taken 56% (This shrinks with longer chains) 2) The chains are statically correlated. Branch probabilities have a very U-shaped distribution. [http://nrs.harvard.edu/urn-3:HUL.InstRepos:24015805] If the branches in a chain are likely to be from the same side of the distribution as their predecessor, but are independent at runtime, this transformation is profitable. (Because the cost of being wrong is a small fixed cost, unlike the standard triangle layout where the cost of being wrong scales with the # of triangles.) 3) The chains are dynamically correlated. If the probability that a previous branch was taken positively influences whether the next branch will be taken We believe that 2 and 3 are common enough to justify the small margin in 1. The code pre-scans a function's CFG to identify this pattern and marks the edges so that the standard layout algorithm can use the computed results. llvm-svn: 296845
* [DAGCombiner] Fix DebugLoc propagation when folding !(x cc y) -> (x !cc y)Taewook Oh2017-03-021-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Currently, when 't1: i1 = setcc t2, t3, cc' followed by 't4: i1 = xor t1, Constant:i1<-1>' is folded into 't5: i1 = setcc t2, t3 !cc', SDLoc of newly created SDValue 't5' follows SDLoc of 't4', not 't1'. However, as the opcode of newly created SDValue is 'setcc', it make more sense to take DebugLoc from 't1' than 't4'. For the code below ``` extern int bar(); extern int baz(); int foo(int x, int y) { if (x != y) return bar(); else return baz(); } ``` , following is the bitcode representation of 'foo' at the end of llvm-ir level optimization: ``` define i32 @foo(i32 %x, i32 %y) !dbg !4 { entry: tail call void @llvm.dbg.value(metadata i32 %x, i64 0, metadata !9, metadata !11), !dbg !12 tail call void @llvm.dbg.value(metadata i32 %y, i64 0, metadata !10, metadata !11), !dbg !13 %cmp = icmp ne i32 %x, %y, !dbg !14 br i1 %cmp, label %if.then, label %if.else, !dbg !16 if.then: ; preds = %entry %call = tail call i32 (...) @bar() #3, !dbg !17 br label %return, !dbg !18 if.else: ; preds = %entry %call1 = tail call i32 (...) @baz() #3, !dbg !19 br label %return, !dbg !20 return: ; preds = %if.else, %if.then %retval.0 = phi i32 [ %call, %if.then ], [ %call1, %if.else ] ret i32 %retval.0, !dbg !21 } !14 = !DILocation(line: 5, column: 9, scope: !15) !16 = !DILocation(line: 5, column: 7, scope: !4) ``` As you can see, in 'entry' block, 'icmp' instruction and 'br' instruction have different debug locations. However, with current implementation, there's no distinction between debug locations of these two when they are lowered to asm instructions. This is because 'icmp' and 'br' become 'setcc' 'xor' and 'brcond' in SelectionDAG, where SDLoc of 'setcc' follows the debug location of 'icmp' but SDLOC of 'xor' and 'brcond' follows the debug location of 'br' instruction, and SDLoc of 'xor' overwrites SDLoc of 'setcc' when they are folded. This patch addresses this issue. Reviewers: atrick, bogner, andreadb, craig.topper, aprantl Reviewed By: andreadb Subscribers: jlebar, mkuper, jholewinski, andreadb, llvm-commits Differential Revision: https://reviews.llvm.org/D29813 llvm-svn: 296825
* [DAG] early exit to improve readability and formatting of visitMemCmpCall(); ↵Sanjay Patel2017-03-021-64/+53
| | | | | | NFCI llvm-svn: 296824
* CodeGen: MachineBlockPlacement: Remove the unused outlining heuristic.Kyle Butt2017-03-021-98/+1
| | | | | | | Outlining optional branches isn't a good heuristic, and it's never been on by default. Remove it to clean things up. llvm-svn: 296818
* [Support] Move Stream library from MSF -> Support.Zachary Turner2017-03-021-2/+2
| | | | | | | | | | After several smaller patches to get most of the core improvements finished up, this patch is a straight move and header fixup of the source. Differential Revision: https://reviews.llvm.org/D30266 llvm-svn: 296810
* [DAG] improve documentation comments; NFCSanjay Patel2017-03-022-90/+48
| | | | llvm-svn: 296808
* Fix some Wdocumentation warningsSimon Pilgrim2017-03-021-2/+2
| | | | llvm-svn: 296783
* [DAGCombiner] avoid assertion when folding binops with opaque constantsSanjay Patel2017-03-021-3/+4
| | | | | | | | | | | | | This bug was introduced with: https://reviews.llvm.org/rL296699 There may be a way to loosen the restriction, but for now just bail out on any opaque constant. The tests show that opacity is target-specific. This goes back to cost calculations in ConstantHoisting based on TTI->getIntImmCost(). llvm-svn: 296768
* fix typo in comment; NFCSanjay Patel2017-03-021-1/+1
| | | | llvm-svn: 296760
* Do not verify MachimeDominatorTree if it is not calculatedSerge Pavlov2017-03-021-14/+14
| | | | | | | | | | | | | | | | If dominator tree is not calculated or is invalidated, set corresponding pointer in the pass state to nullptr. Such pointer value will indicate that operations with dominator tree are not allowed. In particular, it allows to skip verification for such pass state. The dominator tree is not calculated if the machine dominator pass was skipped, it occures in the case of entities with linkage available_externally. The change fixes some test fails observed when expensive checks are enabled. Differential Revision: https://reviews.llvm.org/D29280 llvm-svn: 296742
* LiveRegMatrix: Fix some subreg interference checksMatthias Braun2017-03-021-5/+8
| | | | | | | | Surprisingly, one of the three interference checks in LiveRegMatrix was using the main live range instead of the apropriate subregister range resulting in unnecessarily conservative results. llvm-svn: 296722
* Remove spurious use of LLVM_FALLTHROUGH (NFC)Paul Robinson2017-03-011-43/+17
| | | | llvm-svn: 296713
* [DAGCombiner] mulhi + 1 never overflow.Amaury Sechet2017-03-011-0/+13
| | | | | | | | | | | | | | | Summary: This can be used to optimize large multiplications after legalization. Depends on D29565 Reviewers: mkuper, spatel, RKSimon, zvi, bkramer, aaboud, craig.topper Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D29587 llvm-svn: 296711
* [GlobalISel] Add a way for targets to enable GISel.Ahmed Bougacha2017-03-012-4/+11
| | | | | | | | | | | | | | | | | | | | | | | Until now, we've had to use -global-isel to enable GISel. But using that on other targets that don't support it will result in an abort, as we can't build a full pipeline. Additionally, we want to experiment with enabling GISel by default for some targets: we can't just enable GISel by default, even among those target that do have some support, because the level of support varies. This first step adds an override for the target to explicitly define its level of support. For AArch64, do that using a new command-line option (I know..): -aarch64-enable-global-isel-at-O=<N> Where N is the opt-level below which GISel should be used. Default that to -1, so that we still don't enable GISel anywhere. We're not there yet! While there, remove a couple LLVM_UNLIKELYs. Building the pipeline is such a cold path that in practice that shouldn't matter at all. llvm-svn: 296710
* [DAGCombiner] fold binops with constant into select-of-constantsSanjay Patel2017-03-011-0/+112
| | | | | | | | | | | | | | | | | | This is part of the ongoing attempt to improve select codegen for all targets and select canonicalization in IR (see D24480 for more background). The transform is a subset of what is done in InstCombine's FoldOpIntoSelect(). I first noticed a regression in the x86 avx512-insert-extract.ll tests with a patch that hopes to convert more selects to basic math ops. This appears to be a general missing DAG transform though, so I added tests for all standard binops in rL296621 (PowerPC was chosen semi-randomly; it has scripted FileCheck support, but so do ARM and x86). The poor output for "sel_constants_shl_constant" is tracked with: https://bugs.llvm.org/show_bug.cgi?id=32105 Differential Revision: https://reviews.llvm.org/D30502 llvm-svn: 296699
* [DebugInfo] [DWARFv5] Unique abbrevs for DIEs with different implicit_const ↵Victor Leschuk2017-03-011-2/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | values Take DW_FORM_implicit_const attribute value into account when profiling DIEAbbrevData. Currently if we have two similar types with implicit_const attributes and different values we end up with only one abbrev in .debug_abbrev section. For example consider two structures: S1 with implicit_const attribute ATTR and value VAL1 and S2 with implicit_const ATTR and value VAL2. The .debug_abbrev section will contain only 1 related record: [N] DW_TAG_structure_type DW_CHILDREN_yes DW_AT_ATTR DW_FORM_implicit_const VAL1 // .... This is incorrect as struct S2 (with VAL2) will use abbrev record with VAL1. With this patch we will have two different abbreviations here: [N] DW_TAG_structure_type DW_CHILDREN_yes DW_AT_ATTR DW_FORM_implicit_const VAL1 // .... [M] DW_TAG_structure_type DW_CHILDREN_yes DW_AT_ATTR DW_FORM_implicit_const VAL2 // .... llvm-svn: 296691
* [DAGCombiner] Remove non-ascii character and reflow comment.Benjamin Kramer2017-03-011-5/+4
| | | | llvm-svn: 296690
* LIU:::Query: Query LiveRange instead of LiveInterval; NFCMatthias Braun2017-03-013-16/+15
| | | | | | | | | - We only need the information from the base class, not the additional details in the LiveInterval class. - Spread more `const` - Some code cleanup llvm-svn: 296684
* Elide argument copies during instruction selectionReid Kleckner2017-03-013-9/+219
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Avoids tons of prologue boilerplate when arguments are passed in memory and left in memory. This can happen in a debug build or in a release build when an argument alloca is escaped. This will dramatically affect the code size of x86 debug builds, because X86 fast isel doesn't handle arguments passed in memory at all. It only handles the x86_64 case of up to 6 basic register parameters. This is implemented by analyzing the entry block before ISel to identify copy elision candidates. A copy elision candidate is an argument that is used to fully initialize an alloca before any other possibly escaping uses of that alloca. If an argument is a copy elision candidate, we set a flag on the InputArg. If the the target generates loads from a fixed stack object that matches the size and alignment requirements of the alloca, the SelectionDAG builder will delete the stack object created for the alloca and replace it with the fixed stack object. The load is left behind to satisfy any remaining uses of the argument value. The store is now dead and is therefore elided. The fixed stack object is also marked as mutable, as it may now be modified by the user, and it would be invalid to rematerialize the initial load from it. Supersedes D28388 Fixes PR26328 Reviewers: chandlerc, MatzeB, qcolombet, inglorion, hans Subscribers: igorb, llvm-commits Differential Revision: https://reviews.llvm.org/D29668 llvm-svn: 296683
* LIU::Query: Remove always false member+getter; NFCMatthias Braun2017-03-011-2/+0
| | | | llvm-svn: 296675
* Improve scheduling with branch coalescingNemanja Ivanovic2017-03-014-0/+764
| | | | | | | | | | | This patch adds a MachineSSA pass that coalesces blocks that branch on the same condition. Committing on behalf of Lei Huang. Differential Revision: https://reviews.llvm.org/D28249 llvm-svn: 296670
* [DAG] Prevent Stale nodes from entering worklistNirav Dave2017-03-011-4/+10
| | | | | | | | | | | | | | | Add check that deleted nodes do not get added to worklist. This can occur when a node's operand is simplified to an existing node. This fixes PR32108. Reviewers: jyknight, hfinkel, chandlerc Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D30506 llvm-svn: 296668
* Alphabetize some cases (NFC)Paul Robinson2017-03-011-12/+12
| | | | llvm-svn: 296655
* [DWARF] Default lower bound should respect requested DWARF version.Paul Robinson2017-03-011-10/+22
| | | | | | | | | | | | | DWARF may define a default lower-bound for arrays in languages defined in a particular DWARF version. But the logic to suppress an unnecessary lower-bound attribute was looking at the hard-coded default DWARF version, not the version that had been requested. Also updated the list with all languages defined in DWARF v5. Differential Revision: http://reviews.llvm.org/D30484 llvm-svn: 296652
* [DAGCombiner] Support {a|s}ext, {a|z|s}ext load nodes in load combineArtur Pilipenko2017-03-011-8/+19
| | | | | | | | | | | | Resubmit r295336 after the bug with non-zero offset patterns on BE targets is fixed (r296336). Support {a|s}ext, {a|z|s}ext load nodes as a part of load combine patters. Reviewed By: filcab Differential Revision: https://reviews.llvm.org/D29591 llvm-svn: 296651
* [CodeGen] Remove dead FastISel code after SDAG emitted a tailcall.Ahmed Bougacha2017-03-011-0/+6
| | | | | | | | | | | | | | | | | | When SDAGISel (top-down) selects a tail-call, it skips the remainder of the block. If, before that, FastISel (bottom-up) selected some of the (no-op) next few instructions, we can end up with dead instructions following the terminator (selected by SDAGISel). We need to erase them, as we know they aren't necessary (in addition to being incorrect). We already do this when FastISel falls back on the tail-call itself. Also remove the FastISel-emitted code if we fallback on the instructions between the tail-call and the return. llvm-svn: 296552
* [GlobalISel] Replace all combined G_EXTRACT uses.Ahmed Bougacha2017-03-011-4/+1
| | | | | | | | | | | Iterating on the use-list we're modifying doesn't work: after the first iteration, the use-list iterator will point to a MachineOperand referencing the new register. This caused us to skip the other uses to replace. Instead, use MRI.replaceRegWith(), which accounts for this behavior. llvm-svn: 296551
* Add missing module/license header. NFC.Paul Robinson2017-03-011-0/+13
| | | | llvm-svn: 296550
* [DWARFv5] Emit new unit header format.Paul Robinson2017-02-283-9/+30
| | | | | | | | | Requesting DWARF v5 will now get you the new compile-unit and type-unit headers. llvm-dwarfdump will also recognize them. Differential Revision: http://reviews.llvm.org/D30206 llvm-svn: 296514
* [DAGCombiner] use dyn_cast values in foldSelectOfConstants(); NFCSanjay Patel2017-02-281-6/+8
| | | | llvm-svn: 296502
* [DAGISel] When checking if chain node is foldable, make sure the ↵Craig Topper2017-02-281-1/+1
| | | | | | | | intermediate nodes have a single use across all results not just the result that was used to reach the chain node. This recovers a test case that was severely broken by r296476, my making sure we don't create ADD/ADC that loads and stores when there is also a flag dependency. llvm-svn: 296486
* [Stack Protection] Add diagnostic information for why stack protection was ↵David Bozier2017-02-281-1/+25
| | | | | | | | | | | | | | applied to a function Stack Smash Protection is not completely free, so in hot code, the overhead it causes can cause performance issues. By adding diagnostic information for which functions have SSP and why, a user can quickly determine what they can do to stop SSP being applied to a specific hot function. This change adds a remark that is reported by the stack protection code when an instruction or attribute is encountered that causes SSP to be applied. Patch by: James Henderson Differential Revision: https://reviews.llvm.org/D29023 llvm-svn: 296483
* Revert r296474 - [globalisel] Change LLT constructor string into an LLT ↵Daniel Sanders2017-02-282-23/+56
| | | | | | | | subclass that knows how to generate it. There's a circular dependency that's only revealed when LLVM_ENABLE_MODULES=1. llvm-svn: 296478
* In visitSTORE, always use FindBetterChain, rather than only when UseAA is ↵Nirav Dave2017-02-282-370/+374
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | enabled. Recommiting after fixup of 32-bit aliasing sign offset bug in DAGCombiner. * Simplify Consecutive Merge Store Candidate Search Now that address aliasing is much less conservative, push through simplified store merging search and chain alias analysis which only checks for parallel stores through the chain subgraph. This is cleaner as the separation of non-interfering loads/stores from the store-merging logic. When merging stores search up the chain through a single load, and finds all possible stores by looking down from through a load and a TokenFactor to all stores visited. This improves the quality of the output SelectionDAG and the output Codegen (save perhaps for some ARM cases where we correctly constructs wider loads, but then promotes them to float operations which appear but requires more expensive constant generation). Some minor peephole optimizations to deal with improved SubDAG shapes (listed below) Additional Minor Changes: 1. Finishes removing unused AliasLoad code 2. Unifies the chain aggregation in the merged stores across code paths 3. Re-add the Store node to the worklist after calling SimplifyDemandedBits. 4. Increase GatherAllAliasesMaxDepth from 6 to 18. That number is arbitrary, but seems sufficient to not cause regressions in tests. 5. Remove Chain dependencies of Memory operations on CopyfromReg nodes as these are captured by data dependence 6. Forward loads-store values through tokenfactors containing {CopyToReg,CopyFromReg} Values. 7. Peephole to convert buildvector of extract_vector_elt to extract_subvector if possible (see CodeGen/AArch64/store-merge.ll) 8. Store merging for the ARM target is restricted to 32-bit as some in some contexts invalid 64-bit operations are being generated. This can be removed once appropriate checks are added. This finishes the change Matt Arsenault started in r246307 and jyknight's original patch. Many tests required some changes as memory operations are now reorderable, improving load-store forwarding. One test in particular is worth noting: CodeGen/PowerPC/ppc64-align-long-double.ll - Improved load-store forwarding converts a load-store pair into a parallel store and a memory-realized bitcast of the same value. However, because we lose the sharing of the explicit and implicit store values we must create another local store. A similar transformation happens before SelectionDAG as well. Reviewers: arsenm, hfinkel, tstellarAMD, jyknight, nhaehnle llvm-svn: 296476
* [globalisel] Change LLT constructor string into an LLT subclass that knows ↵Daniel Sanders2017-02-282-56/+23
| | | | | | | | | | | | | | | | | | how to generate it. Summary: This will allow future patches to inspect the details of the LLT. The implementation is now split between the Support and CodeGen libraries to allow TableGen to use this class without introducing layering concerns. Thanks to Ahmed Bougacha for finding a reasonable way to avoid the layering issue and providing the version of this patch without that problem. Reviewers: t.p.northover, qcolombet, rovka, aditya_nandakumar, ab, javed.absar Subscribers: arsenm, nhaehnle, mgorny, dberris, llvm-commits, kristof.beyls Differential Revision: https://reviews.llvm.org/D30046 llvm-svn: 296474
* [ImplicitNullCheck] Add alias analysis usageSanjoy Das2017-02-281-27/+75
| | | | | | | | | | | | | | | | | | | Summary: With this change ImplicitNullCheck optimization uses alias analysis and can use load/store memory access for implicit null check if there are other load/store before but memory accesses do not alias. Patch by Serguei Katkov! Reviewers: sanjoy Reviewed By: sanjoy Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D30331 llvm-svn: 296440
* Revert "Add MIR-level outlining pass"Matthias Braun2017-02-284-1407/+0
| | | | | | | | Revert Machine Outliner for now, as it breaks the asan bot. This reverts commit r296418. llvm-svn: 296426
* Add MIR-level outlining passMatthias Braun2017-02-284-0/+1407
| | | | | | | | | | | | | | | | | | | | | | | | | | This is a patch for the outliner described in the RFC at: http://lists.llvm.org/pipermail/llvm-dev/2016-August/104170.html The outliner is a code-size reduction pass which works by finding repeated sequences of instructions in a program, and replacing them with calls to functions. This is useful to people working in low-memory environments, where sacrificing performance for space is acceptable. This adds an interprocedural outliner directly before printing assembly. For reference on how this would work, this patch also includes X86 target hooks and an X86 test. The outliner is run like so: clang -mno-red-zone -mllvm -enable-machine-outliner file.c Patch by Jessica Paquette<jpaquette@apple.com>! rdar://29166825 Differential Revision: https://reviews.llvm.org/D26872 llvm-svn: 296418
OpenPOWER on IntegriCloud