summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen
Commit message (Collapse)AuthorAgeFilesLines
...
* [CGP] Split some critical edges coming out of indirect branchesMichael Kuperstein2017-02-281-0/+157
| | | | | | | | | | | | | | | | | | | | | | Splitting critical edges when one of the source edges is an indirectbr is hard in general (because it requires changing the memory the indirectbr reads). But if a block only has a single indirectbr predecessor (which is the common case), we can simulate splitting that edge by splitting the destination block, and retargeting the *direct* branches. This is motivated by the use of computed gotos in python 2.7: PyEval_EvalFrame() ends up using an indirect branch with ~100 successors, and passing a constant to each of those. Since MachineSink can't break indirect critical edges on demand (and doing this in MIR doesn't look feasible), this causes us to emit about ~100 defs of registers containing constants, which we in the predecessor block, where only one of those constants is used in each successor. So, at each computed goto, we needlessly spill about a 100 constants to stack. The end result is that a clang-compiled python interpreter can be about ~2.5x slower on a simple python reduction loop than a gcc-compiled interpreter. Differential Revision: https://reviews.llvm.org/D29916 llvm-svn: 296416
* [PDB] Make streams carry their own endianness.Zachary Turner2017-02-281-1/+1
| | | | | | | | | | | | | Before the endianness was specified on each call to read or write of the StreamReader / StreamWriter, but in practice it's extremely rare for streams to have data encoded in multiple different endiannesses, so we should optimize for the 99% use case. This makes the code cleaner and more general, but otherwise has NFC. llvm-svn: 296415
* [CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; ↵Eugene Zelenko2017-02-276-31/+97
| | | | | | other minor fixes (NFC). llvm-svn: 296404
* ISel: We need to notify FastIS of the IMPLICIT_DEF we created in ↵Arnold Schwaighofer2017-02-271-1/+7
| | | | | | | | | | createSwiftErrorEntriesInEntryBlock Otherwise, it will insert instructions before it. rdar://30536186 llvm-svn: 296395
* [PDB] Partial resubmit of r296215, which improved PDB Stream Library.Zachary Turner2017-02-271-3/+2
| | | | | | | | | | | | | | | | | This was reverted because it was breaking some builds, and because of incorrect error code usage. Since the CL was large and contained many different things, I'm resubmitting it in pieces. This portion is NFC, and consists of: 1) Renaming classes to follow a consistent naming convention. 2) Fixing the const-ness of the interface methods. 3) Adding detailed doxygen comments. 4) Fixing a few instances of passing `const BinaryStream& X`. These are now passed as `BinaryStreamRef X`. llvm-svn: 296394
* Revert "DAG: Check if extract_vector_elt is legal or custom"Matt Arsenault2017-02-271-1/+1
| | | | | | | This reverts r295782. This could potentially result in some legalization loops and I avoided the need for this. llvm-svn: 296393
* [X86][SSE] Attempt to extract vector elements through target shufflesSimon Pilgrim2017-02-271-0/+15
| | | | | | | | | | DAGCombiner already supports peeking thorough shuffles to improve vector element extraction, but legalization often leaves us in situations where we need to extract vector elements after shuffles have already been lowered. This patch adds support for VECTOR_EXTRACT_ELEMENT/PEXTRW/PEXTRB instructions to attempt to handle target shuffles as well. I've covered some basic scenarios including handling shuffle mask scaling and the implicit zero-extension of PEXTRW/PEXTRB, there is more that could be done here (that I've mentioned in TODOs) but I haven't found many cases where its worth it. Differential Revision: https://reviews.llvm.org/D30176 llvm-svn: 296381
* [TailDuplicator] Maintain DebugLoc for branch instructionsTaewook Oh2017-02-271-1/+2
| | | | | | | | | | | | | | Summary: Existing implementation of duplicateSimpleBB function drops DebugLoc metadata of branch instructions during the transformation. This patch addresses this issue by making newly created branch instructions to keep the metadata of replaced branch instructions. Reviewers: qcolombet, craig.topper, aprantl, MatzeB, sanjoy, dblaikie Reviewed By: dblaikie Subscribers: dblaikie, llvm-commits Differential Revision: https://reviews.llvm.org/D30026 llvm-svn: 296371
* [DAGCombine] Fix for a load combine bug with non-zero offset patterns on BE ↵Artur Pilipenko2017-02-271-0/+4
| | | | | | | | | | | | | | | | | | targets This pattern is essentially a i16 load from p+1 address: %p1.i16 = bitcast i8* %p to i16* %p2.i8 = getelementptr i8, i8* %p, i64 2 %v1 = load i16, i16* %p1.i16 %v2.i8 = load i8, i8* %p2.i8 %v2 = zext i8 %v2.i8 to i16 %v1.shl = shl i16 %v1, 8 %res = or i16 %v1.shl, %v2 Current implementation would identify %v1 load as the first byte load and would mistakenly emit a i16 load from %p1.i16 address. This patch adds a check that the first byte is loaded from a non-zero offset of the first load address. This way this address can be used as the base address for the combined value. Otherwise just give up combining. llvm-svn: 296336
* [DAGCombine] NFC. MatchLoadCombine extract MemoryByteOffset lambda helperArtur Pilipenko2017-02-271-9/+13
| | | | | | This refactoring will simplify the upcoming change to fix the bug in folding patterns with non-zero offsets on BE targets. llvm-svn: 296332
* [DAGCombine] NFC. MatchLoadCombine remember the first byte provider, not the ↵Artur Pilipenko2017-02-271-3/+5
| | | | | | | | load node This refactoring will simplify the upcoming change to fix a bug in folding patterns with non-zero offsets on BE targets. llvm-svn: 296331
* Revert "[CGP] Split some critical edges coming out of indirect branches"Daniel Jasper2017-02-261-155/+0
| | | | | | | This reverts commit r296149 as it leads to crashes when compiling for PPC. llvm-svn: 296295
* Revert "In visitSTORE, always use FindBetterChain, rather than only when ↵Nirav Dave2017-02-262-374/+370
| | | | | | | | UseAA is enabled." This reverts commit r296252 until 256-bit operations are more efficiently generated in X86. llvm-svn: 296279
* [ExecutionDepsFix] Don't make copies of LiveReg objects when collecting ↵Craig Topper2017-02-251-13/+8
| | | | | | | | | | | | | | | | | | | | | | | operands for soft instructions Summary: While collecting operands we make copies of the LiveReg objects which are stored in the LiveRegs array. If the instruction uses the same register multiple times we end up with multiple copies. Later we iterate through the collected list of LiveReg objects and merge DomainValues. In the process of doing this the merge function can change the contents of the original LiveReg object in the LiveRegs array, but not the copies that have been made. So when we get to the second usage of the register we end up seeing a stale copy of the LiveReg object. To fix this I've stopped copying and now just store a pointer to the original LiveReg object. Another option might be to avoid adding the same register to the Regs array twice, but this approach seemed simpler. The included test case exposes this bug due to an AVX-512 masked OR instruction using the same register for the passthru operand and one of the inputs to the OR operation. Fixes PR30284. Reviewers: RKSimon, stoklund, MatzeB, spatel, myatsina Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D30242 llvm-svn: 296260
* No need to copy the variable [NFC]Artyom Skrobov2017-02-251-2/+1
| | | | llvm-svn: 296259
* Revert r296215, "[PDB] General improvements to Stream library." and followings.NAKAMURA Takumi2017-02-251-2/+3
| | | | | | | | | | | | | | | | | r296215, "[PDB] General improvements to Stream library." r296217, "Disable BinaryStreamTest.StreamReaderObject temporarily." r296220, "Re-enable BinaryStreamTest.StreamReaderObject." r296244, "[PDB] Disable some tests that are breaking bots." r296249, "Add static_cast to silence -Wc++11-narrowing." std::errc::no_buffer_space should be used for OS-oriented errors for socket transmission. (Seek discussions around llvm/xray.) I could substitute s/no_buffer_space/others/g, but I revert whole them ATM. Could we define and use LLVM errors there? llvm-svn: 296258
* In visitSTORE, always use FindBetterChain, rather than only when UseAA is ↵Nirav Dave2017-02-252-370/+374
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | enabled. Recommiting after fixup of 32-bit aliasing sign offset bug in DAGCombiner. * Simplify Consecutive Merge Store Candidate Search Now that address aliasing is much less conservative, push through simplified store merging search and chain alias analysis which only checks for parallel stores through the chain subgraph. This is cleaner as the separation of non-interfering loads/stores from the store-merging logic. When merging stores search up the chain through a single load, and finds all possible stores by looking down from through a load and a TokenFactor to all stores visited. This improves the quality of the output SelectionDAG and the output Codegen (save perhaps for some ARM cases where we correctly constructs wider loads, but then promotes them to float operations which appear but requires more expensive constant generation). Some minor peephole optimizations to deal with improved SubDAG shapes (listed below) Additional Minor Changes: 1. Finishes removing unused AliasLoad code 2. Unifies the chain aggregation in the merged stores across code paths 3. Re-add the Store node to the worklist after calling SimplifyDemandedBits. 4. Increase GatherAllAliasesMaxDepth from 6 to 18. That number is arbitrary, but seems sufficient to not cause regressions in tests. 5. Remove Chain dependencies of Memory operations on CopyfromReg nodes as these are captured by data dependence 6. Forward loads-store values through tokenfactors containing {CopyToReg,CopyFromReg} Values. 7. Peephole to convert buildvector of extract_vector_elt to extract_subvector if possible (see CodeGen/AArch64/store-merge.ll) 8. Store merging for the ARM target is restricted to 32-bit as some in some contexts invalid 64-bit operations are being generated. This can be removed once appropriate checks are added. This finishes the change Matt Arsenault started in r246307 and jyknight's original patch. Many tests required some changes as memory operations are now reorderable, improving load-store forwarding. One test in particular is worth noting: CodeGen/PowerPC/ppc64-align-long-double.ll - Improved load-store forwarding converts a load-store pair into a parallel store and a memory-realized bitcast of the same value. However, because we lose the sharing of the explicit and implicit store values we must create another local store. A similar transformation happens before SelectionDAG as well. Reviewers: arsenm, hfinkel, tstellarAMD, jyknight, nhaehnle llvm-svn: 296252
* Minor code cleanup. NFC.Junmo Park2017-02-251-1/+1
| | | | llvm-svn: 296222
* [PDB] General improvements to Stream library.Zachary Turner2017-02-251-3/+2
| | | | | | | | | | | | | | | This adds various new functionality and cleanup surrounding the use of the Stream library. Major changes include: * Renaming of all classes for more consistency / meaningfulness * Addition of some new methods for reading multiple values at once. * Full suite of unit tests for reader / writer functionality. * Full set of doxygen comments for all classes. * Streams now store their own endianness. * Fixed some bugs in a few of the classes that were discovered by the unit tests. llvm-svn: 296215
* [PDB] Rename Stream related source files.Zachary Turner2017-02-251-2/+2
| | | | | | | | | | | | | | This is part of a larger effort to get the Stream code moved up to Support. I don't want to do it in one large patch, in part because the changes are so big that it will treat everything as file deletions and add, losing history in the process. Aside from that though, it's just a good idea in general to make small changes. So this change only changes the names of the Stream related source files, and applies necessary source fix ups. llvm-svn: 296211
* [WebAssembly] Add support for using a wasm global for the stack pointer.Dan Gohman2017-02-241-0/+1
| | | | | | | This replaces the __stack_pointer variable which was allocated in linear memory. llvm-svn: 296201
* [WebAssembly] Basic support for Wasm object file encoding.Dan Gohman2017-02-241-3/+58
| | | | | | | | | With the "wasm32-unknown-unknown-wasm" triple, this allows writing out simple wasm object files, and is another step in a larger series toward migrating from ELF to general wasm object support. Note that this code and the binary format itself is still experimental. llvm-svn: 296190
* Revert "Correct register pressure calculation in presence of subregs"Stanislav Mekhanoshin2017-02-243-42/+13
| | | | | | | | This reverts commit r296009. It broke one out of tree target and also does not account for all partial lines added or removed when calculating PressureDiff. llvm-svn: 296182
* GlobalISel: check for CImm rather than Imm on G_CONSTANTs.Tim Northover2017-02-241-2/+5
| | | | | | | All G_CONSTANTS created by the MachineIRBuilder have an operand of type CImm (i.e. a ConstantInt), so that's what the selector needs to look for. llvm-svn: 296176
* [CodeGenPrepare] Make -addr-sink-using-gep work with address spaces.Eli Friedman2017-02-241-4/+4
| | | | | | | | | | When we construct addressing modes, we use isNoopAddrSpaceCast to ignore addrspacecast instructions. Make sure we insert the correct addrspacecast when we reconstruct the addressing mode. Differential Revision: https://reviews.llvm.org/D30114 llvm-svn: 296167
* [CGP] Split some critical edges coming out of indirect branchesMichael Kuperstein2017-02-241-0/+155
| | | | | | | | | | | | | | | | | | | | | | Splitting critical edges when one of the source edges is an indirectbr is hard in general (because it requires changing the memory the indirectbr reads). But if a block only has a single indirectbr predecessor (which is the common case), we can simulate splitting that edge by splitting the destination block, and retargeting the *direct* branches. This is motivated by the use of computed gotos in python 2.7: PyEval_EvalFrame() ends up using an indirect branch with ~100 successors, and passing a constant to each of those. Since MachineSink can't break indirect critical edges on demand (and doing this in MIR doesn't look feasible), this causes us to emit about ~100 defs of registers containing constants, which we in the predecessor block, where only one of those constants is used in each successor. So, at each computed goto, we needlessly spill about a 100 constants to stack. The end result is that a clang-compiled python interpreter can be about ~2.5x slower on a simple python reduction loop than a gcc-compiled interpreter. Differential Revision: https://reviews.llvm.org/D29916 llvm-svn: 296149
* [DAGCombiner] add missing folds for scalar select of {-1,0,1}Sanjay Patel2017-02-241-3/+32
| | | | | | | | | | | | | | | | | | | | | | | | The motivation for filling out these select-of-constants cases goes back to D24480, where we discussed removing an IR fold from add(zext) --> select. And that goes back to: https://reviews.llvm.org/rL75531 https://reviews.llvm.org/rL159230 The idea is that we should always canonicalize patterns like this to a select-of-constants in IR because that's the smallest IR and the best for value tracking. Note that we currently do the opposite in some cases (like the cases in *this* patch). Ie, the proposed folds in this patch already exist in InstCombine today: https://github.com/llvm-mirror/llvm/blob/master/lib/Transforms/InstCombine/InstCombineSelect.cpp#L1151 As this patch shows, most targets generate better machine code for simple ext/add/not ops rather than a select of constants. So the follow-up steps to make this less of a patchwork of special-case folds and missing IR canonicalization: 1. Have DAGCombiner convert any select of constants into ext/add/not ops. 2 Have InstCombine canonicalize in the other direction (create more selects). Differential Revision: https://reviews.llvm.org/D30180 llvm-svn: 296137
* [globalisel] Decouple src pattern operands from dst pattern operands.Daniel Sanders2017-02-241-0/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This isn't testable for AArch64 by itself so this patch also adds support for constant immediates in the pattern and physical register uses in the result. The new IntOperandMatcher matches the constant in patterns such as '(set $rd:GPR32, (G_XOR $rs:GPR32, -1))'. It's always safe to fold immediates into an instruction so this is the first rule that will match across multiple BB's. The Renderer hierarchy is responsible for adding operands to the result instruction. Renderers can copy operands (CopyRenderer) or add physical registers (in particular %wzr and %xzr) to the result instruction in any order (OperandMatchers now import the operand names from SelectionDAG to allow renderers to access any operand). This allows us to emit the result instruction for: %1 = G_XOR %0, -1 --> %1 = ORNWrr %wzr, %0 %1 = G_XOR -1, %0 --> %1 = ORNWrr %wzr, %0 although the latter is untested since the matcher/importer has not been taught about commutativity yet. Added BuildMIAction which can build new instructions and mutate them where possible. W.r.t the mutation aspect, MatchActions are now told the name of an instruction they can recycle and BuildMIAction will emit mutation code when the renderers are appropriate. They are appropriate when all operands are rendered using CopyRenderer and the indices are the same as the matcher. This currently assumes that all operands have at least one matcher. Finally, this change also fixes a crash in AArch64InstructionSelector::select() caused by an immediate operand passing isImm() rather than isCImm(). This was uncovered by the other changes and was detected by existing tests. Depends on D29711 Reviewers: t.p.northover, ab, qcolombet, rovka, aditya_nandakumar, javed.absar Reviewed By: rovka Subscribers: aemerson, dberris, kristof.beyls, llvm-commits Differential Revision: https://reviews.llvm.org/D29712 llvm-svn: 296131
* Add missing initialization for MachineOptimizationRemarkEmitterJustin Bogner2017-02-241-0/+1
| | | | | | This was missed in r293110. llvm-svn: 296096
* [ExecutionDepsFix] Use range-based for loop. NFCCraig Topper2017-02-241-2/+1
| | | | llvm-svn: 296093
* Revert r269060 to pacify bots.Michael Kuperstein2017-02-241-155/+0
| | | | llvm-svn: 296064
* [CGP] Split some critical edges coming out of indirect branchesMichael Kuperstein2017-02-241-0/+155
| | | | | | | | | | | | | | | | | | | | | | Splitting critical edges when one of the source edges is an indirectbr is hard in general (because it requires changing the memory the indirectbr reads). But if a block only has a single indirectbr predecessor (which is the common case), we can simulate splitting that edge by splitting the destination block, and retargeting the *direct* branches. This is motivated by the use of computed gotos in python 2.7: PyEval_EvalFrame() ends up using an indirect branch with ~100 successors, and passing a constant to each of those. Since MachineSink can't break indirect critical edges on demand (and doing this in MIR doesn't look feasible), this causes us to emit about ~100 defs of registers containing constants, which we in the predecessor block, where only one of those constants is used in each successor. So, at each computed goto, we needlessly spill about a 100 constants to stack. The end result is that a clang-compiled python interpreter can be about ~2.5x slower on a simple python reduction loop than a gcc-compiled interpreter. Differential Revision: https://reviews.llvm.org/D29916 llvm-svn: 296060
* [GlobalISel] Use the same name for all remarks.Ahmed Bougacha2017-02-241-2/+2
| | | | | | While there, switch to the explicit ctor. llvm-svn: 296059
* [GlobalISel] Use the DISubprogram for translation failure remarks.Ahmed Bougacha2017-02-242-3/+5
| | | | | | | Justin added support for DISubprogram locs in r295531 and r296052. Use that instead of no-loc for constants and arguments. llvm-svn: 296058
* [GlobalISel] Remove now-unnecessary variable. NFC.Ahmed Bougacha2017-02-241-15/+14
| | | | | | | Since r296047, we're able to return early on failures. Don't track whether we succeeded. llvm-svn: 296057
* OptDiag: Summarize the instruction count in asm-printerJustin Bogner2017-02-241-1/+13
| | | | | | | Add an optimization remark to asm-printer that summarizes the number of instructions emitted per function. llvm-svn: 296053
* [GlobalISel] Don't translate other blocks when one failed.Ahmed Bougacha2017-02-231-35/+33
| | | | | | | | | | We were stopping the translation of the parent block when the translation of an instruction failed, but we were still trying to translate the other blocks of the parent function. Don't do that. llvm-svn: 296047
* [GlobalISel] Finalize translated function on scope exit. NFC.Ahmed Bougacha2017-02-231-3/+4
| | | | | | | This is the compromise between having a per-function IRTranslator and manually managing the per-function state. llvm-svn: 296046
* CodeGen: MachineBlockPlacement: Rename member to more general name. NFC.Kyle Butt2017-02-231-13/+11
| | | | | | | | | Rename ComputedTrellisEdges to ComputedEdges to allow for other methods of pre-computing edges. Differential Revision: https://reviews.llvm.org/D30308 llvm-svn: 296018
* [GlobalISel] Emit opt remarks on isel fallbacks.Ahmed Bougacha2017-02-235-81/+111
| | | | | | | | | | | | | | | | | Having more fine-grained information on the specific construct that caused us to fallback is valuable for large-scale data collection. We still have the fallback warning, that's also used for FastISel. We still need to remove the fallback warning, and teach FastISel to also emit remarks (it currently has a combination of the warning, stats, and debug prints: the remarks could unify all three). The abort-on-fallback path could also be better handled using remarks: one could imagine a "-Rpass-error", analoguous to "-Werror", which would promote missed/failed remarks to errors. It's not clear whether that would be useful for other remarks though, so we're not there yet. llvm-svn: 296013
* [CodeGen] Teach opt remarks how to print MI instructions.Ahmed Bougacha2017-02-231-0/+10
| | | | | | This will be used with GISel opt remarks. llvm-svn: 296012
* [CodeGen] Print MI without a newline when skipping debugloc. NFC.Ahmed Bougacha2017-02-231-1/+3
| | | | | | | This matches the behavior for skip-operands. While there, document it. This is a follow-up to r296007. llvm-svn: 296011
* Correct register pressure calculation in presence of subregsStanislav Mekhanoshin2017-02-233-13/+42
| | | | | | | | | | If a subreg is used in an instruction it counts as a whole superreg for the purpose of register pressure calculation. This patch corrects improper register pressure calculation by examining operand's lane mask. Differential Revision: https://reviews.llvm.org/D29835 llvm-svn: 296009
* [CodeGen] Add a way to SkipDebugLoc in MachineInstr::print(). NFC.Ahmed Bougacha2017-02-231-4/+5
| | | | llvm-svn: 296007
* [GlobalISel] Simplify Select type cleanup using a ScopeExit. NFC.Ahmed Bougacha2017-02-231-13/+18
| | | | | | This lets us use more natural early-returns when selection fails. llvm-svn: 296006
* [DAG] add convenience function to get -1 constant; NFCISanjay Patel2017-02-231-32/+15
| | | | llvm-svn: 296004
* [LazyMachineBFI] Reimplement with getAnalysisIfAvailableAdam Nemet2017-02-232-21/+44
| | | | | | | | | | | | | | | | | | | | | | | | | Since LoopInfo is not available in machine passes as universally as in IR passes, using the same approach for OptimizationRemarkEmitter as we did for IR will run LoopInfo and DominatorTree unnecessarily. (LoopInfo is not used lazily by ORE.) To fix this, I am modifying the approach I took in D29836. LazyMachineBFI now uses its client passes including MachineBFI itself that are available or otherwise compute them on the fly. So for example GreedyRegAlloc, since it's already using MBFI, will reuse that instance. On the other hand, AsmPrinter in Justin's patch will generate DT, LI and finally BFI on the fly. (I am of course wondering now if the simplicity of this approach is even preferable in IR. I will do some experiments.) Testing is provided by an updated version of D29837 which requires Justin's patch to bring ORE to the AsmPrinter. Differential Revision: https://reviews.llvm.org/D30128 llvm-svn: 295996
* Fix signed/unsigned comparison warning on MSVCSimon Pilgrim2017-02-231-1/+1
| | | | llvm-svn: 295962
* [CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; ↵Eugene Zelenko2017-02-225-74/+160
| | | | | | other minor fixes (NFC). llvm-svn: 295893
* [DAGCombiner] revert r295336Bill Seurer2017-02-221-19/+8
| | | | | | | | | | | r295336 causes a bootstrapped clang to fail for many compilations on powerpc BE. See http://lab.llvm.org:8011/builders/clang-ppc64be-linux-multistage/builds/2315 for example. Reverting as per the developer's request. llvm-svn: 295849
OpenPOWER on IntegriCloud