summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen
Commit message (Collapse)AuthorAgeFilesLines
...
* Second attempt to add iterator_range::empty()Jordan Rose2019-10-076-8/+8
| | | | | | | | | | | | Doing this makes MSVC complain that `empty(someRange)` could refer to either C++17's std::empty or LLVM's llvm::empty, which previously we avoided via SFINAE because std::empty is defined in terms of an empty member rather than begin and end. So, switch callers over to the new method as it is added. https://reviews.llvm.org/D68439 llvm-svn: 373935
* [FPEnv] Add constrained intrinsics for lrint and lroundKevin P. Neal2019-10-075-28/+88
| | | | | | | | | | | Earlier in the year intrinsics for lrint, llrint, lround and llround were added to llvm. The constrained versions are now implemented here. Reviewed by: andrew.w.kaylor, craig.topper, cameron.mcinally Approved by: craig.topper Differential Revision: https://reviews.llvm.org/D64746 llvm-svn: 373900
* [X86][AVX] Access a scalar float/double as a free extract from a broadcast ↵Simon Pilgrim2019-10-061-0/+5
| | | | | | | | | | | | | | | | load (PR43217) If a fp scalar is loaded and then used as both a scalar and a vector broadcast, perform the load as a broadcast and then extract the scalar for 'free' from the 0th element. This involved switching the order of the X86ISD::BROADCAST combines so we only convert to X86ISD::BROADCAST_LOAD once all other canonicalizations have been attempted. Adds a DAGCombinerInfo::recursivelyDeleteUnusedNodes wrapper. Fixes PR43217 Differential Revision: https://reviews.llvm.org/D68544 llvm-svn: 373871
* [LegalizeTypes][X86] When splitting a vselect for type legalization, don't ↵Craig Topper2019-10-061-3/+12
| | | | | | | | | | | | | | | | | | split a setcc condition if the setcc input is legal and vXi1 conditions are supported Summary: The VSELECT splitting code tries to split a setcc input as well. But on avx512 where mask registers are well supported it should be better to just split the mask and use a single compare. Reviewers: RKSimon, spatel, efriedma Reviewed By: spatel Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68359 llvm-svn: 373863
* Revert [DAGCombine] Match more patterns for half word bswapSanjay Patel2019-10-061-29/+29
| | | | | | | | This reverts r373850 (git commit 25ba49824d2d4f2347b4a7cb1623600a76ce9433) This patch appears to cause multiple codegen regression test failures - http://lab.llvm.org:8011/builders/clang-cmake-armv7-quick/builds/10680 llvm-svn: 373853
* [DAGCombine] Match more patterns for half word bswapAmaury Sechet2019-10-061-29/+29
| | | | | | | | | | | | | | Summary: It ensures that the bswap is generated even when a part of the subtree already matches a bswap transform. Reviewers: craig.topper, efriedma, RKSimon, lebedev.ri Subscribers: llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68250 llvm-svn: 373850
* GlobalISel: Partially implement lower for G_EXTRACTMatt Arsenault2019-10-061-0/+35
| | | | | | Turn into shift and truncate. Doesn't yet handle pointers. llvm-svn: 373838
* [FastISel] Copy the inline assembly dialect to the INLINEASM instruction.Craig Topper2019-10-051-0/+1
| | | | | | Fixes PR43575. llvm-svn: 373836
* BranchFolding - IsBetterFallthrough - assert non-null pointers. NFCI.Simon Pilgrim2019-10-051-0/+2
| | | | | | Silences static analyzer null dereference warnings. llvm-svn: 373823
* Fix a *nasty* miscompile in experimental unordered atomic loweringPhilip Reames2019-10-051-3/+4
| | | | | | | | | | This is an omission in rL371441. Loads which happened to be unordered weren't being added to the PendingLoad set, and thus weren't be ordered w/respect to side effects which followed before the end of the block. Included test case is how I spotted this. We had an atomic load being folded into a using instruction after a fence that load was supposed to be ordered with. I'm sure it showed up a bunch of other ways as well. Spotted via manual inspecting of assembly differences in a corpus w/and w/o the new experimental mode. Finding this with testing would have been "unpleasant". llvm-svn: 373814
* Revert [CodeGen] Do the Simple Early Return in block-placement pass to ↵Reid Kleckner2019-10-041-46/+0
| | | | | | | | | | | | | | optimize the blocks This reverts r371177 (git commit f879c6875563c0a8cd838f1e13b14dd33558f1f8) It caused PR43566 by removing empty, address-taken MachineBasicBlocks. Such blocks may have references from blockaddress or other operands, and need more consideration to be removed. See the PR for a test case to use when relanding. llvm-svn: 373805
* [MachineOutliner] Disable outlining from noreturn functionsJessica Paquette2019-10-041-0/+6
| | | | | | | | | | | | | | | | Outlining from noreturn functions doesn't do the correct thing right now. The outliner should respect that the caller is marked noreturn. In the event that we have a noreturn function, and the outlined code is in tail position, the outliner will not see that the outlined function should be tail called. As a result, you end up with a regular call containing a return. Fixing this requires that we check that all candidates live inside noreturn functions. So, for the sake of correctness, don't outline from noreturn functions right now. Add machine-outliner-noreturn.mir to test this. llvm-svn: 373791
* [ScheduleDAG] When a node is cloned, add an edge between the nodes.Eli Friedman2019-10-041-0/+4
| | | | | | | | | | | | | | | InstrEmitter's virtual register handling assumes that clones are emitted after the cloned node. Make sure this assumption actually holds. Fixes a "Node emitted out of order - early" assertion on the testcase. This is probably a very rare case to actually hit in practice; even without the explicit edge, the scheduler will usually end up scheduling the nodes in the expected order due to other constraints. Differential Revision: https://reviews.llvm.org/D68068 llvm-svn: 373782
* [ModuloSchedule] Do not remap terminatorsJames Molloy2019-10-041-1/+1
| | | | | | | | | | This is a trivial point fix. Terminator instructions aren't scheduled, so we shouldn't expect to be able to remap them. This doesn't affect Hexagon and PPC because their terminators are always hardware loop backbranches that have no register operands. llvm-svn: 373762
* Fix MSVC "not all control paths return a value" warning. NFCI.Simon Pilgrim2019-10-041-0/+1
| | | | llvm-svn: 373741
* [DebugInfo] LiveDebugValues: move DBG_VALUE creation into VarLoc classJeremy Morse2019-10-041-107/+137
| | | | | | | | | | | | | | | | | | | | | | Rather than having a mixture of location-state shared between DBG_VALUEs and VarLoc objects in LiveDebugValues, this patch makes VarLoc the master record of variable locations. The refactoring means that the transfer of locations from one place to another is always a performed by an operation on an existing VarLoc, that produces another transferred VarLoc. DBG_VALUEs are only created at the end of LiveDebugValues, once all locations are known. As a plus, there is now only one method where DBG_VALUEs can be created. The test case added covers a circumstance that is now impossible to express in LiveDebugValues: if an already-indirect DBG_VALUE is spilt, previously it would have been restored-from-spill as a direct DBG_VALUE. We now don't lose this information along the way, as VarLocs always refer back to the "original" non-transfer DBG_VALUE, and we can always work out whether a location was "originally" indirect. Differential Revision: https://reviews.llvm.org/D67398 llvm-svn: 373727
* [DebugInfo] LiveDebugValues: defer DBG_VALUE creation during analysisJeremy Morse2019-10-041-8/+7
| | | | | | | | | | | | | | | | | | When transfering variable locations from one place to another, LiveDebugValues immediately creates a DBG_VALUE representing that transfer. This causes trouble if the variable location should subsequently be invalidated by a loop back-edge, such as in the added test case: the transfer DBG_VALUE from a now-invalid location is used as proof that the variable location is correct. This is effectively a self-fulfilling prophesy. To avoid this, defer the insertion of transfer DBG_VALUEs until after analysis has completed. Some of those transfers are still sketchy, but we don't propagate them into other blocks now. Differential Revision: https://reviews.llvm.org/D67393 llvm-svn: 373720
* [DAGCombiner] add operation legality checks before creating shift ops (PR43542)Sanjay Patel2019-10-031-1/+6
| | | | | | | | | | | | | | As discussed on llvm-dev and: https://bugs.llvm.org/show_bug.cgi?id=43542 ...we have transforms that assume shift operations are legal and transforms to use them are profitable, but that may not hold for simple targets. In this case, the MSP430 target custom lowers shifts by repeating (many) simpler/fixed ops. That can be avoided by keeping this code as setcc/select. Differential Revision: https://reviews.llvm.org/D68397 llvm-svn: 373666
* DebugInfo: Generalize rnglist emission as a precursor to reusing it for ↵David Blaikie2019-10-031-15/+25
| | | | | | loclist emission llvm-svn: 373663
* [ModuloSchedule] removeBranch() *before* creating the trip count conditionJames Molloy2019-10-031-2/+1
| | | | | | | | | | | | | | The Hexagon code assumes there's no existing terminator when inserting its trip count condition check. This causes swp-stages5.ll to break. The generated code looks good to me, it is likely a permutation. I have disabled the new codegen path to keep everything green and will investigate along with the other 3-4 tests that have different codegen. Fixes expensive-checks build. llvm-svn: 373629
* [Alignment][NFC] Remove StoreInst::setAlignment(unsigned)Guillaume Chatelet2019-10-031-1/+1
| | | | | | | | | | | | | | | | | Summary: This is patch is part of a series to introduce an Alignment type. See this thread for context: http://lists.llvm.org/pipermail/llvm-dev/2019-July/133851.html See this patch for the introduction of the type: https://reviews.llvm.org/D64790 Reviewers: courbet, bollu, jdoerfert Subscribers: hiraditya, asbirlea, cfe-commits, llvm-commits Tags: #clang, #llvm Differential Revision: https://reviews.llvm.org/D68268 llvm-svn: 373595
* DebugInfo: Rename DebugLocStream::Entry::Begin/EndSym to just Begin/EndDavid Blaikie2019-10-022-17/+13
| | | | | | | | Brings this struct in line with the RangeSpan class so they might eventually be used by common template code for generating range/loc lists with less duplicate code. llvm-svn: 373540
* [LegalizeTypes] Check for already split condition before calilng ↵Craig Topper2019-10-021-4/+4
| | | | | | | | SplitVecRes_SETCC in SplitRes_SELECT. No point in manually splitting the SETCC if it was already done. llvm-svn: 373535
* DebugInfo: Simplify RangeSpan to be a plain structDavid Blaikie2019-10-023-21/+15
| | | | | | | | | | | | | | This is an effort to make RangeSpan and DebugLocStream::Entry more similar to share code for their emission (to reuse the more complicated code for using (& choosing when to use) base address selection entries, etc). It didn't seem like this struct was worth the complexity of encapsulation - when the members could be initialized by the ctor to any value (no validation) and the type is assignable (so there's no mutability or other constraint being implemented by its interface). llvm-svn: 373533
* [CodeGen] Remove unused MachineMemOperand::print wrappers (PR41772)Simon Pilgrim2019-10-021-11/+0
| | | | | | | | | | As noted on PR41772, the static analyzer reports that the MachineMemOperand::print partial wrappers set a number of args to null pointers that were then dereferenced in the actual implementation. It turns out that these wrappers are not being used at all (hence why we're not seeing any crashes), so I'd like to propose we just get rid of them. Differential Revision: https://reviews.llvm.org/D68208 llvm-svn: 373484
* Reapply r373431 "Switch lowering: omit range check for bit tests when ↵Hans Wennborg2019-10-021-18/+25
| | | | | | | | | | | | | | | | | default is unreachable (PR43129)" This was reverted in r373454 due to breaking the expensive-checks bot. This version addresses that by omitting the addSuccessorWithProb() call when omitting the range check. > Switch lowering: omit range check for bit tests when default is unreachable (PR43129) > > This is modeled after the same functionality for jump tables, which was > added in r357067. > > Differential revision: https://reviews.llvm.org/D68131 llvm-svn: 373477
* AsmPrinter - emitGlobalConstantFP - silence static analyzer null dereference ↵Simon Pilgrim2019-10-021-5/+2
| | | | | | | | warning. NFCI. All the calls to emitGlobalConstantFP should provide a nonnull Type for the float. llvm-svn: 373464
* [ModuloSchedule] Peel out prologs and epilogs, generate actual codeJames Molloy2019-10-024-4/+396
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This extends the PeelingModuloScheduleExpander to generate prolog and epilog code, and correctly stitch uses through the prolog, kernel, epilog DAG. The key concept in this patch is to ensure that all transforms are *local*; only a function of a block and its immediate predecessor and successor. By defining the problem in this way we can inductively rewrite the entire DAG using only local knowledge that is easy to reason about. For example, we assume that all prologs and epilogs are near-perfect clones of the steady-state kernel. This means that if a block has an instruction that is predicated out, we can redirect all users of that instruction to that equivalent instruction in our immediate predecessor. As all blocks are clones, every instruction must have an equivalent in every other block. Similarly we can make the assumption by construction that if a value defined in a block is used outside that block, the only possible user is its immediate successors. We maintain this even for values that are used outside the loop by creating a limited form of LCSSA. This code isn't small, but it isn't complex. Enabled a bunch of testing from Hexagon. There are a couple of tests not enabled yet; I'm about 80% sure there isn't buggy codegen but the tests are checking for patterns that we don't produce. Those still need a bit more investigation. In the meantime we (Google) are happy with the code produced by this on our downstream SMS implementation, and believe it generates correct code. Subscribers: mgorny, hiraditya, jsji, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68205 llvm-svn: 373462
* Revert r373431 "Switch lowering: omit range check for bit tests when default ↵Hans Wennborg2019-10-021-23/+17
| | | | | | | | | | | | | | | is unreachable (PR43129)" This broke http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-win/builds/19967 > Switch lowering: omit range check for bit tests when default is unreachable (PR43129) > > This is modeled after the same functionality for jump tables, which was > added in r357067. > > Differential revision: https://reviews.llvm.org/D68131 llvm-svn: 373454
* WinException::emitExceptHandlerTable - silence static analyzer ↵Simon Pilgrim2019-10-021-2/+1
| | | | | | | | dyn_cast<Function> null dereference warning. NFCI. The static analyzer is warning about a potential null dereference, but we should be able to use cast<Function> directly and if not assert will fire for us. llvm-svn: 373449
* Switch lowering: omit range check for bit tests when default is unreachable ↵Hans Wennborg2019-10-021-17/+23
| | | | | | | | | | | (PR43129) This is modeled after the same functionality for jump tables, which was added in r357067. Differential revision: https://reviews.llvm.org/D68131 llvm-svn: 373431
* DebugInfo: Update support for detecting C++ language variants in debug info ↵David Blaikie2019-10-022-4/+5
| | | | | | emission llvm-svn: 373420
* [Dominators][CodeGen] Don't mark MachineDominatorTree as preserved in ↵Jakub Kuderski2019-10-011-1/+0
| | | | | | MachineLICM llvm-svn: 373378
* [Dominators][CodeGen] Fix MachineDominatorTree preservation in PHIEliminationJakub Kuderski2019-10-012-2/+8
| | | | | | | | | | | | | | | | | | | Summary: PHIElimination modifies CFG and marks MachineDominatorTree as preserved. Therefore, it the CFG changes it should also update the MDT, when available. This patch teaches PHIElimination to recalculate MDT when necessary. This fixes the `tailmerging_in_mbp.ll` test failure discovered after switching to generic DomTree verification algorithm in MachineDominators in D67976. Reviewers: arsenm, hliao, alex-t, rampitec, vpykhtin, grosser Reviewed By: rampitec Subscribers: MatzeB, wdng, hiraditya, javed.absar, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68154 llvm-svn: 373377
* Reapply [Dominators][CodeGen] Clean up MachineDominatorsJakub Kuderski2019-10-011-13/+3
| | | | | | | | This reverts r373117 (git commit 159ef37735f21ae373282e0c53cbd9b6af1e0dfd) Phabricator review: https://reviews.llvm.org/D67976. llvm-svn: 373376
* [AMDGPU] Add VerifyScheduling support.Jay Foad2019-10-011-3/+4
| | | | | | | | | | | | | | | | Summary: This is cut and pasted from the corresponding GenericScheduler functions. Reviewers: arsenm, atrick, tstellar, vpykhtin Subscribers: MatzeB, kzhuravl, jvesely, wdng, nhaehnle, yaxunl, dstuttard, tpr, t-tye, hiraditya, javed.absar, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68264 llvm-svn: 373346
* [DAG][X86] Convert isNegatibleForFree/GetNegatedExpression to a target hook ↵Simon Pilgrim2019-10-012-276/+272
| | | | | | | | | | | | | | | | (PR42863) This patch converts the DAGCombine isNegatibleForFree/GetNegatedExpression into overridable TLI hooks. The intention is to let us extend existing FNEG combines to work more generally with negatible float ops, allowing it work with target specific combines and opcodes (e.g. X86's FMA variants). Unlike the SimplifyDemandedBits, we can't just handle target nodes through a Target callback, we need to do this as an override to allow targets to handle generic opcodes as well. This does mean that the target implementations has to duplicate some checks (recursion depth etc.). Partial reversion of rL372756 - I've identified the infinite loop issue inside the X86 override but haven't fixed it yet so I've only (re)committed the common TargetLowering refactoring part of the patch. Differential Revision: https://reviews.llvm.org/D67557 llvm-svn: 373343
* [Dominators][CodeGen] Add MachinePostDominatorTree verificationJakub Kuderski2019-10-013-8/+17
| | | | | | | | | | | | | | | | | | | | | Summary: This patch implements Machine PostDominator Tree verification and ensures that the verification doesn't fail the in-tree tests. MPDT verification can be enabled using `verify-machine-dom-info` -- the same flag used by Machine Dominator Tree verification. Flipping the flag revealed that MachineSink falsely claimed to preserve CFG and MDT/MPDT. This patch fixes that. Reviewers: arsenm, hliao, rampitec, vpykhtin, grosser Reviewed By: hliao Subscribers: wdng, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68235 llvm-svn: 373341
* Revert "GlobalISel: Handle llvm.read_register"Dmitri Gribenko2019-10-011-15/+0
| | | | | | | | This reverts commit r373294. It broke Clang's CodeGen/arm64-microsoft-status-reg.cpp: http://lab.llvm.org:8011/builders/clang-x86_64-debian-fast/builds/18483 llvm-svn: 373310
* GlobalISel: Handle llvm.read_registerMatt Arsenault2019-10-011-0/+15
| | | | | | | | | | | | | SelectionDAG has a bunch of machinery to defer this to selection time for some reason. Just directly emit a copy during IRTranslator. The x86 usage does somewhat questionably check hasFP, which could depend on the whole function being at minimum translated. This does lose the convergent bit if the callsite had it, which may be a problem. We also lose that in general for intrinsics, which may also be a problem. llvm-svn: 373294
* TLI: Remove DAG argument from getRegisterByNameMatt Arsenault2019-10-011-4/+4
| | | | | | | | | | | Replace with the MachineFunction. X86 is the only user, and only uses it for the function. This removes one obstacle from using this in GlobalISel. The other is the more tolerable EVT argument. The X86 use of the function seems questionable to me. It checks hasFP, before frame lowering. llvm-svn: 373292
* GlobalISel: Implement widenScalar for G_SITOFP/G_UITOFP sourcesMatt Arsenault2019-10-011-4/+6
| | | | | | Legalize 16-bit G_SITOFP/G_UITOFP for AMDGPU. llvm-svn: 373287
* DebugInfo: Simplify section label caching/usageDavid Blaikie2019-09-302-9/+4
| | | | llvm-svn: 373273
* [DAGCombiner] Clang format MatchRotate. NFCAmaury Sechet2019-09-301-4/+6
| | | | llvm-svn: 373269
* [globalisel][knownbits] Allow targets to call ↵Daniel Sanders2019-09-302-3/+5
| | | | | | | | | | | | | | | | | | GISelKnownBits::computeKnownBitsImpl() Summary: It seems we missed that the target hook can't query the known-bits for the inputs to a target instruction. Fix that oversight Reviewers: aditya_nandakumar Subscribers: rovka, hiraditya, volkan, Petar.Avramovic, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D67380 llvm-svn: 373264
* [DAGCombiner] Update MatchRotate so that it returns an SDValue. NFCAmaury Sechet2019-09-301-22/+21
| | | | llvm-svn: 373260
* [NewPM] Port MachineModuleInfo to the new pass manager.Yuanfang Chen2019-09-3011-60/+105
| | | | | | | | | | | | | Existing clients are converted to use MachineModuleInfoWrapperPass. The new interface is for defining a new pass manager API in CodeGen. Reviewers: fedor.sergeev, philip.pfaffe, chandlerc, arsenm Reviewed By: arsenm, fedor.sergeev Differential Revision: https://reviews.llvm.org/D64183 llvm-svn: 373240
* [AArch64][GlobalISel] Support lowering variadic musttail callsJessica Paquette2019-09-301-0/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | This adds support for lowering variadic musttail calls. To do this, we have to... - Detect a musttail call in a variadic function before attempting to lower the call's formal arguments. This is done in the IRTranslator. - Compute forwarded registers in `lowerFormalArguments`, and add copies for those registers. - Restore the forwarded registers in `lowerTailCall`. Because there doesn't seem to be any nice way to wrap these up into the outgoing argument handler, the restore code in `lowerTailCall` is done separately. Also, irritatingly, you have to make sure that the registers don't overlap with any passed parameters. Otherwise, the scheduler doesn't know what to do with the extra copies and asserts. Add call-translator-variadic-musttail.ll to test this. This is pretty much the same as the X86 musttail-varargs.ll test. We didn't have as nice of a test to base this off of, but the idea is the same. Differential Revision: https://reviews.llvm.org/D68043 llvm-svn: 373226
* [SSP] [3/3] cmpxchg and addrspacecast instructions can nowPaul Robinson2019-09-301-2/+21
| | | | | | | | | | | | | | | | trigger stack protectors. Fixes PR42238. Add test coverage for llvm.memset, as proxy for all llvm.mem* intrinsics. There are two issues here: (1) they could be lowered to a libc call, which could be intercepted, and do Bad Stuff; (2) with a non-constant size, they could overwrite the current stack frame. The test was mostly written by Matt Arsenault in r363169, which was later reverted; I tweaked what he had and added the llvm.memset part. Differential Revision: https://reviews.llvm.org/D67845 llvm-svn: 373220
* [SSP] [2/3] Refactor an if/dyn_cast chain to switch on opcode. NFCPaul Robinson2019-09-301-15/+24
| | | | | | Differential Revision: https://reviews.llvm.org/D67844 llvm-svn: 373219
OpenPOWER on IntegriCloud