summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen
Commit message (Collapse)AuthorAgeFilesLines
...
* [ARM GlobalISel] Legalize G_SITOFP and G_UITOFPDiana Picus2018-01-301-0/+19
| | | | | | | | Legal if we have hardware support, libcall otherwise. Also add supporting code to the legalizer helper for libcalls. llvm-svn: 323730
* [ARM GlobalISel] Legalize G_FPTOSI and G_FPTOUIDiana Picus2018-01-301-0/+18
| | | | | | | | | Legal if we have hardware support for floating point, libcalls otherwise. Also add the necessary support for libcalls in the legalizer helper. llvm-svn: 323726
* [SelectionDAG]: Ignore "returned" in the presence of an implicit sret.Dan Gohman2018-01-301-2/+4
| | | | | | | | | | | | | | | | When a function return value can't be directly lowered, such as returning an i128 on WebAssembly, as indicated by the CanLowerReturn target hook, SelectionDAGBuilder can translate it to return the value through a hidden sret-like argument. If such a function has an argument with the "returned" attribute, the attribute can't be automatically lowered, because the function no longer has a normal return value. For now, just discard the "returned" attribute. This fixes PR36128. llvm-svn: 323715
* [RAFast] Don't dereference MBB::endQuentin Colombet2018-01-291-5/+6
| | | | | | | | | | | | | | | | When RAFast sees liveins in on a basic block, it uses that information to initialize the availability of the registers. The called method uses an instruction as one of its argument and in the liveins case, RAFast was dereferencing MBB::begin which can be MBB::end for empty basic block. Change the API of definePhysReg to use MachineBasicBlock::iterator instead of MachineInstr so that we don't dereference an invalid iterator while making the call. rdar://problem/36952401 llvm-svn: 323710
* LiveInterval: Print weight in print() function.Matthias Braun2018-01-291-0/+1
| | | | llvm-svn: 323702
* [ARM][GISel] PR35965 Constrain RegClasses of nested instructions built from ↵Daniel Sanders2018-01-291-0/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Dst Pattern Summary: Apparently, we missed on constraining register classes of VReg-operands of all the instructions built from a destination pattern but the root (top-level) one. The issue exposed itself while selecting G_FPTOSI for armv7: the corresponding pattern generates VTOSIZS wrapped into COPY_TO_REGCLASS, so top-level COPY_TO_REGCLASS gets properly constrained, while nested VTOSIZS (or rather its destination virtual register to be exact) does not. Fixing this by issuing GIR_ConstrainSelectedInstOperands for every nested GIR_BuildMI. https://bugs.llvm.org/show_bug.cgi?id=35965 rdar://problem/36886530 Patch by Roman Tereshin Reviewers: dsanders, qcolombet, rovka, bogner, aditya_nandakumar, volkan Reviewed By: dsanders, qcolombet, rovka Subscribers: aemerson, javed.absar, kristof.beyls, llvm-commits Differential Revision: https://reviews.llvm.org/D42565 llvm-svn: 323692
* [globalisel][legalizer] Change identity() to changeTo() to clarify that it ↵Daniel Sanders2018-01-291-1/+1
| | | | | | | | | | | | | changes things. NFC Prior to committing r323681, we decided to change pick() to identity() since it wasn't clear from the name what pick() did. However, identity() isn't a very good name either since it implies that no changes are made. For some reason, naming it changeTo() didn't occur to me until just after the commit. This should resolve the lack of clarity that pick() had while still implying that it changes the MIR. llvm-svn: 323689
* [CodeGen] Simplify conditional. NFCShoaib Meenai2018-01-291-1/+1
| | | | | | | | | | | Rafael pointed out that `hasInternalLinkage() || hasPrivateLinkage()` is equivalent to `hasLocalLinkage()` in post-commit review. I'm intentionally not updating the comment, partly because I like it being explicit, and partly because "global symbols with local linkage" sounds like an oxymoron. llvm-svn: 323688
* [globalisel][legalizer] Adapt LegalizerInfo to support inter-type ↵Daniel Sanders2018-01-294-6/+228
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | dependencies and other things. Summary: As discussed in D42244, we have difficulty describing the legality of some operations. We're not able to specify relationships between types. For example, declaring the following setAction({..., 0, s32}, Legal) setAction({..., 0, s64}, Legal) setAction({..., 1, s32}, Legal) setAction({..., 1, s64}, Legal) currently declares these type combinations as legal: {s32, s32} {s64, s32} {s32, s64} {s64, s64} but we currently have no means to say that, for example, {s64, s32} is not legal. Some operations such as G_INSERT/G_EXTRACT/G_MERGE_VALUES/ G_UNMERGE_VALUES have relationships between the types that are currently described incorrectly. Additionally, G_LOAD/G_STORE currently have no means to legalize non-atomics differently to atomics. The necessary information is in the MMO but we have no way to use this in the legalizer. Similarly, there is currently no way for the register type and the memory type to differ so there is no way to cleanly represent extending-load/truncating-store in a way that can't be broken by optimizers (resulting in illegal MIR). It's also difficult to control the legalization strategy. We've added support for legalizing non-power of 2 types but there's still some hardcoded assumptions about the strategy. The main one I've noticed is that type0 is always legalized before type1 which is not a good strategy for `type0 = G_EXTRACT type1, ...` if you need to widen the container. It will converge on the same result eventually but it will take a much longer route when legalizing type0 than if you legalize type1 first. Lastly, the definition of legality and the legalization strategy is kept separate which is not ideal. It's helpful to be able to look at a one piece of code and see both what is legal and the method the legalizer will use to make illegal MIR more legal. This patch adds a layer onto the LegalizerInfo (to be removed when all targets have been migrated) which resolves all these issues. Here are the rules for shift and division: for (unsigned BinOp : {G_LSHR, G_ASHR, G_SDIV, G_UDIV}) getActionDefinitions(BinOp) .legalFor({s32, s64}) // If type0 is s32/s64 then it's Legal .clampScalar(0, s32, s64) // If type0 is <s32 then WidenScalar to s32 // If type0 is >s64 then NarrowScalar to s64 .widenScalarToPow2(0) // Round type0 scalars up to powers of 2 .unsupported(); // Otherwise, it's unsupported This describes everything needed to both define legality and describe how to make illegal things legal. Here's an example of a complex rule: getActionDefinitions(G_INSERT) .unsupportedIf([=](const LegalityQuery &Query) { // If type0 is smaller than type1 then it's unsupported return Query.Types[0].getSizeInBits() <= Query.Types[1].getSizeInBits(); }) .legalIf([=](const LegalityQuery &Query) { // If type0 is s32/s64/p0 and type1 is a power of 2 other than 2 or 4 then it's legal // We don't need to worry about large type1's because unsupportedIf caught that. const LLT &Ty0 = Query.Types[0]; const LLT &Ty1 = Query.Types[1]; if (Ty0 != s32 && Ty0 != s64 && Ty0 != p0) return false; return isPowerOf2_32(Ty1.getSizeInBits()) && (Ty1.getSizeInBits() == 1 || Ty1.getSizeInBits() >= 8); }) .clampScalar(0, s32, s64) .widenScalarToPow2(0) .maxScalarIf(typeInSet(0, {s32}), 1, s16) // If type0 is s32 and type1 is bigger than s16 then NarrowScalar type1 to s16 .maxScalarIf(typeInSet(0, {s64}), 1, s32) // If type0 is s64 and type1 is bigger than s32 then NarrowScalar type1 to s32 .widenScalarToPow2(1) // Round type1 scalars up to powers of 2 .unsupported(); This uses a lambda to say that G_INSERT is unsupported when type0 is bigger than type1 (in practice, this would be a default rule for G_INSERT). It also uses one to describe the legal cases. This particular predicate is equivalent to: .legalFor({{s32, s1}, {s32, s8}, {s32, s16}, {s64, s1}, {s64, s8}, {s64, s16}, {s64, s32}}) In terms of performance, I saw a slight (~6%) performance improvement when AArch64 was around 30% ported but it's pretty much break even right now. I'm going to take a look at constexpr as a means to reduce the initialization cost. Future work: * Make it possible for opcodes to share rulesets. There's no need for G_LSHR/G_ASHR/G_SDIV/G_UDIV to have separate rule and ruleset objects. There's no technical barrier to this, it just hasn't been done yet. * Replace the type-index numbers with an enum to get .clampScalar(Type0, s32, s64) * Better names for things like .maxScalarIf() (clampMaxScalar?) and the vector rules. * Improve initialization cost using constexpr Possible future work: * It's possible to make these rulesets change the MIR directly instead of returning a description of how to change the MIR. This should remove a little overhead caused by parsing the description and routing to the right code, but the real motivation is that it removes the need for LegalizeAction::Custom. With Custom removed, there's no longer a requirement that Custom legalization change the opcode to something that's considered legal. Reviewers: ab, t.p.northover, qcolombet, rovka, aditya_nandakumar, volkan, reames, bogner Reviewed By: bogner Subscribers: hintonda, bogner, aemerson, mgorny, javed.absar, kristof.beyls, llvm-commits Differential Revision: https://reviews.llvm.org/D42251 llvm-svn: 323681
* [MachineVerifier] Add check that renamable operands aren't reserved registers.Geoff Berry2018-01-291-6/+8
| | | | | | | | | | | | Summary: Reviewers: qcolombet, MatzeB Subscribers: arsenm, sdardis, nhaehnle, mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D42449 llvm-svn: 323676
* [AMDGPU][X86][Mips] Make sure renamable bit not set for reserved regsGeoff Berry2018-01-292-3/+20
| | | | | | | | | Summary: Fix a few places that were modifying code after register allocation to set the renamable bit correctly to avoid failing the validation added in D42449. llvm-svn: 323675
* [globalisel] Make LegalizerInfo::LegalizeAction available outside of ↵Daniel Sanders2018-01-292-11/+14
| | | | | | | | | | | | LegalizerInfo. NFC Summary: The improvements to the LegalizerInfo discussed in D42244 require that LegalizerInfo::LegalizeAction be available for use in other classes. As such, it needs to be moved out of LegalizerInfo. This has been done separately to the next patch to minimize the noise in that patch. llvm-svn: 323669
* [AccelTable] Workaround for MSVC bugJonas Devlieghere2018-01-291-5/+32
| | | | | | | | Microsoft Visual Studio rejects the static constexpr static list of atoms even though it's valid C++. This provides a workaround to unbreak the bots. llvm-svn: 323667
* [AccelTable] Try making MSVC happyJonas Devlieghere2018-01-291-4/+4
| | | | | | | | MSVC complains that the constexpr "expression did not evaluate to a constant". Trying to make it happy by adding a `const` specifier as suggested in https://stackoverflow.com/questions/37574343. llvm-svn: 323659
* [dsymutil] Generate Apple accelerator tablesJonas Devlieghere2018-01-291-0/+14
| | | | | | | | | | | This patch adds support for generating accelerator tables in dsymutil. This feature was already present in our internal repository but not yet upstreamed because it requires changes to the Apple accelerator table implementation. Differential revision: https://reviews.llvm.org/D42501 llvm-svn: 323655
* [NFC] Rename DwarfAccelTable and move header.Jonas Devlieghere2018-01-295-391/+6
| | | | | | | | | | This patch renames DwarfAccelTable.{h,cpp} to AccelTable.{h,cpp} and moves the header to the include dir so it is accessible by the dsymutil implementation. Differential revision: https://reviews.llvm.org/D42529 llvm-svn: 323654
* [NFC] Refactor Apple Accelerator TablesJonas Devlieghere2018-01-294-318/+360
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch refactors the way data is stored in the accelerator table and makes them truly generic. There have been several attempts to do this in the past: - D8215 & D8216: Using a union and partial hardcoding. - D11805: Using inheritance. - D42246: Using a callback. In the end I didn't like either of them, because for some reason or another parts of it felt hacky or decreased runtime performance. I didn't want to completely rewrite them as I was hoping that we could reuse parts for the successor in the DWARF standard. However, it seems less and less likely that there will be a lot of opportunities for sharing code and/or an interface. Originally I choose to template the whole class, because it introduces no performance overhead compared to the original implementation. We ended up settling on a hybrid between a templated method and a virtual call to emit the data. The motivation is that we don't want to increase code size for a feature that should soon be superseded by the DWARFv5 accelerator tables. While the code will continue to be used for compatibility, it won't be on the hot path. Furthermore this does not regress performance compared to Apple's internal implementation that already uses virtual calls for this. A quick summary for why these changes are necessary: dsymutil likes to reuse the current implementation of the Apple accelerator tables. However, LLDB expects a slightly different interface than what is currently emitted. Additionally, in dsymutil we only have offsets and no actual DIEs. Although the patch suggests a lot of code has changed, this change is pretty straightforward: - We created an abstract class `AppleAccelTableData` to serve as an interface for the different data classes. - We created two implementations of this class, one for type tables and one for everything else. There will be a third one for dsymutil that takes just the offset. - We use the supplied class to deduct the atoms for the header which makes the structure of the table fully self contained, although not enforced by the interface as was the case for the fully templated approach. - We renamed the prefix from DWARF- to Apple- to make space for the future implementation of .debug_names. This change is NFC and relies on the existing tests. Differential revision: https://reviews.llvm.org/D42334 llvm-svn: 323653
* [DebugInfo] Fix fragment offset emission order for symbol locationsMikael Holmen2018-01-291-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: When emitting the location for a global variable with fragmented debug expressions, make sure that the offset pieces, which represent optimized-out parts of the variable, are emitted before their succeeding fragments' expressions. Previously, if the succeeding fragment's location was a symbol, the offset piece was emitted after, rather than before, that symbol's expression. This effectively meant that the symbols were associated with the wrong parts of the variable. This fixes PR36085. Patch by: David Stenberg Reviewers: aprantl, probinson, dblaikie Reviewed By: aprantl Subscribers: JDevlieghere, llvm-commits Tags: #debug-info Differential Revision: https://reviews.llvm.org/D42527 llvm-svn: 323644
* [X86] Make foldLogicOfSetCCs work better for vectors pre legal types/operationsCraig Topper2018-01-291-1/+1
| | | | | | | | | | | | | | | | | Summary: There's a check in the code to only check getSetCCResultType after LegalOperations or if the type is MVT::i1. But the i1 check is only allowing scalar types through. I think it should check that the scalar type is MVT::i1 so that it will work for vectors. The changed test already does this combine with AVX512VL where getSetCCResultType returns vXi1. But with avx512f and no VLX getSetCCResultType returns a type matching the width of the input type. Reviewers: spatel, RKSimon Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D42619 llvm-svn: 323631
* [NFC] fix trivial typos in comments and documentsHiroshi Inoue2018-01-291-1/+1
| | | | | | "to to" -> "to" llvm-svn: 323628
* [Support] Move DJB hash to support. NFCJonas Devlieghere2018-01-281-1/+2
| | | | | | | | | | | This patch moves the DJB hash to support. This is consistent with other hashing algorithms living there. The hash is used by the DWARF accelerator tables. We're doing this now because the hashing function is needed by dsymutil and we don't want to link against libBinaryFormat. Differential revision: https://reviews.llvm.org/D42594 llvm-svn: 323616
* [TargetLowering] Teach TargetLowering::SimplifySetCC to simplify setcc of ↵Craig Topper2018-01-271-14/+16
| | | | | | | | vXi1 vectors into logic ops. This transform was already being done for setcc of scalar i1. This extends it to vectors. llvm-svn: 323585
* [SelectionDAG] Make DAGTypeLegalizer::PromoteSetCCOperands handle ↵Craig Topper2018-01-271-4/+4
| | | | | | | | SETEQ/SETNE correctly for vector types. The code was using getValueSizeInBits and combining with the result of a call to DAG.ComputeNumSignBits. But for vector types getValueSizeInBits returns the width of the full vector while ComputeNumSignBits is going to give a number no larger than the width of a single element. So we should be using getScalarValueSizeInBits to get the element width. llvm-svn: 323583
* [GlobalISel][Legalizer] Convert the FP constants to the right APFloat type ↵Amara Emerson2018-01-271-1/+18
| | | | | | | | | | | for G_FCONSTANT. We weren't converting the immediate ConstantFP during legalization, which caused the wrong bit patterns to be emitted for half type FP constants. Fixes PR36106. llvm-svn: 323582
* [LivePhysRegs] Preserve pristine regs in blocks with no successors.Eli Friedman2018-01-261-2/+2
| | | | | | | | | | | | | | | One common source of blocks with no successors is calls to noreturn functions; we want to preserve pristine registers in case they throw an exception. The whole pristine register thing is messy (we should really prefer to explicitly model registers), but this fills a hole in the model for now. Fixes https://bugs.llvm.org/show_bug.cgi?id=36073. Differential Revision: https://reviews.llvm.org/D42509 llvm-svn: 323559
* [SelectionDAGISel] Add a debug print before call to Select. Adjust where ↵Craig Topper2018-01-261-6/+5
| | | | | | | | | | | | blank lines are printed during isel process to make things more sensibly grouped. Previously some targets printed their own message at the start of Select to indicate what they were selecting. For the targets that didn't, it means there was no print of the root node before any custom handling in the target executed. So if the target did something custom and never called SelectNodeCommon, no print would be made. For the targets that did print a message in Select, if they didn't custom handle a node SelectNodeCommon would reprint the root node before walking the isel table. It seems better to just print the message before the call to Select so all targets behave the same. And then remove the root node printing from SelectNodeCommon and just leave a message that says we're starting the table search. There were also some oddities in blank line behavior. Usually due to a \n after a call to SelectionDAGNode::dump which already inserted a new line. llvm-svn: 323551
* [DWARF] Generate DWARF v5 string offsets tables along with strx* index forms.Wolfgang Pieb2018-01-269-38/+181
| | | | | | | | | | | | Summary: This is the producer side for DWARF v5 string offsets tables. The reader/consumer side was committed with r321295. All compile and type units in a module share a contribution to the string offsets table. Indirect strings use the strx{1,2,3,4} index forms. Reviewers: dblaikie, aprantl, JDevliegehere Differential Revision: https://reviews.llvm.org/D42021 llvm-svn: 323546
* [DAG] Teach findBaseOffset to interpret indexes of indexed memory operationsNirav Dave2018-01-261-8/+35
| | | | | | Indexed outputs are addition / subtractions and can be interpreted as such. llvm-svn: 323539
* [DAGCombine] reduceBuildVecToShuffle - ensure EXTRACT_VECTOR_ELT index is in ↵Simon Pilgrim2018-01-261-1/+5
| | | | | | | | range From OSS Fuzz Test Case #5688 llvm-svn: 323535
* [MIR] Add support for addrspace in MIRFrancis Visoiu Mistrih2018-01-264-0/+20
| | | | | | | | | | Add support for printing / parsing the addrspace of a MachineMemOperand. Fixes PR35970. Differential Revision: https://reviews.llvm.org/D42502 llvm-svn: 323521
* [NFC] fix trivial typos in comments and documentsHiroshi Inoue2018-01-264-4/+4
| | | | | | "in in" -> "in", "on on" -> "on" etc. llvm-svn: 323508
* [SelectionDAG] Replace a std::vector<SDValue> with a SmallVector.Craig Topper2018-01-261-1/+1
| | | | | | It likely the number of elements in the type we're legalizing here is reasonably small. llvm-svn: 323505
* [CGP] Re-enable Select in complex addressing mode.Serguei Katkov2018-01-261-1/+1
| | | | | | Switch Select handling on after fixing two bugs: rL323192 and rL323497. llvm-svn: 323498
* [CodeGen] Ignore private symbols in llvm.used for COFFShoaib Meenai2018-01-261-4/+4
| | | | | | | Similar to the existing handling for internal symbols, private symbols are also not visible to the linker and should be ignored. llvm-svn: 323483
* [GISel]: Implement GlobalISel combiner API.Aditya Nandakumar2018-01-253-0/+124
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | https://reviews.llvm.org/D41373 The various components are GICombinerHelper contains transformations that are common to all targets. Targets can pick and choose which transformations (at function/opcode granularity) each pass uses via configuring a GICombinerInfo. GICombiner contains some common code and it does the traversal, driving of combines, worklist management and iterating until convergence. GICombinerInfo is an interface with a virtual method called combine. The combiner info will allow targets to pick and choose (or implement their own specific combines). CombineInfos can make use of available combines in GICombineHelper to configure the transformations for a particular pass. Currently this approach allows cherry picking transformations from helpers (at function/opcode granularity) and also allows early returning on specific transformations. Targets also get to prioritize whether target specific combines run before/after the opt-in generic combines. Ideally we would like this part to be configured by both C++ and Tablegen. The CombinerInfo also has a field which indicates how to deal with IllegalOps (ie - should we allow to create them/or legalize them?). A CombinerPass would configure a CombinerInfo, create the GICombiner with the Info, and call GICombiner::combineMachineInstrs(MachineFunction&). This organization is very similar to the GISelLegalizer. llvm-svn: 323392
* [GlobalISel] Don't fall back to FastISel.Amara Emerson2018-01-242-1/+5
| | | | | | | Apparently checking the pass structure isn't enough to ensure that we don't fall back to FastISel, as it's set up as part of the SelectionDAGISel. llvm-svn: 323369
* [globalisel] Introduce LegalityQuery to better encapsulate the legalizer ↵Daniel Sanders2018-01-242-14/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | decisions. NFC. Summary: `getAction(const InstrAspect &) const` breaks encapsulation by exposing the smaller components that are used to decide how to legalize an instruction. This is a problem because we need to change the implementation of LegalizerInfo so that it's able to describe particular type combinations rather than just cartesian products of types. For example, declaring the following setAction({..., 0, s32}, Legal) setAction({..., 0, s64}, Legal) setAction({..., 1, s32}, Legal) setAction({..., 1, s64}, Legal) currently declares these type combinations as legal: {s32, s32} {s64, s32} {s32, s64} {s64, s64} but we currently have no means to say that, for example, {s64, s32} is not legal. Some operations such as G_INSERT/G_EXTRACT/G_MERGE_VALUES/ G_UNMERGE_VALUES has relationships between the types that are currently described incorrectly. Additionally, G_LOAD/G_STORE currently have no means to legalize non-atomics differently to atomics. The necessary information is in the MMO but we have no way to use this in the legalizer. Similarly, there is currently no way for the register type and the memory type to differ so there is no way to cleanly represent extending-load/truncating-store in a way that can't be broken by optimizers (resulting in illegal MIR). This patch introduces LegalityQuery which provides all the information needed by the legalizer to make a decision on whether something is legal and how to legalize it. Reviewers: ab, t.p.northover, qcolombet, rovka, aditya_nandakumar, volkan, reames, bogner Reviewed By: bogner Subscribers: bogner, llvm-commits, kristof.beyls Differential Revision: https://reviews.llvm.org/D42244 llvm-svn: 323342
* [DebugInfo] Emit DWARF reference for DIVariable 'count' in DISubrangeSander de Smalen2018-01-242-1/+8
| | | | | | | | | | | | | | | | | | | | | | Summary: This patch implements the codegen of DWARF debug info for non-constant 'count' fields for DISubrange. This is patch [2/3] in a series to extend LLVM's DISubrange Metadata node to support debugging of C99 variable length arrays and vectors with runtime length like the Scalable Vector Extension for AArch64. It is also a first step towards representing more complex cases like arrays in Fortran. Reviewers: echristo, pcc, aprantl, dexonsmith, clayborg, kristof.beyls, dblaikie Reviewed By: aprantl Subscribers: fhahn, aemerson, rengolin, JDevlieghere, llvm-commits Differential Revision: https://reviews.llvm.org/D41696 llvm-svn: 323323
* [Metadata] Extend 'count' field of DISubrange to take a metadata nodeSander de Smalen2018-01-242-2/+6
| | | | | | | | | | | | | | | | | | | | | | | Summary: This patch extends the DISubrange 'count' field to take either a (signed) constant integer value or a reference to a DILocalVariable or DIGlobalVariable. This is patch [1/3] in a series to extend LLVM's DISubrange Metadata node to support debugging of C99 variable length arrays and vectors with runtime length like the Scalable Vector Extension for AArch64. It is also a first step towards representing more complex cases like arrays in Fortran. Reviewers: echristo, pcc, aprantl, dexonsmith, clayborg, kristof.beyls, dblaikie Reviewed By: aprantl Subscribers: rnk, probinson, fhahn, aemerson, rengolin, JDevlieghere, llvm-commits Differential Revision: https://reviews.llvm.org/D41695 llvm-svn: 323313
* [DAGCombiner] Bail out if vector size is not a multipleSven van Haastregt2018-01-241-0/+4
| | | | | | | | | | | | For the included test case, the DAG transformation concat_vectors(scalar, undef) -> scalar_to_vector(sclr) would attempt to create a v2i32 vector for a v9i8 concat_vector. Bail out to avoid creating a bitcast with mismatching sizes later on. Differential Revision: https://reviews.llvm.org/D42379 llvm-svn: 323312
* [GlobalMerge] Don't merge dllexport globalsMartin Storsjo2018-01-241-1/+2
| | | | | | | | | Merging such globals loses the dllexport attribute. Add a test to check that normal globals still are merged. Differential Revision: https://reviews.llvm.org/D42127 llvm-svn: 323307
* [GISel]: Remove redundant copies at the end of ISelAditya Nandakumar2018-01-241-0/+32
| | | | | | | | | https://reviews.llvm.org/D42402 A lot of these copies are useless (copies b/w VRegs having the same regclass) and should be cleaned up. llvm-svn: 323291
* [safestack] Inline safestack pointer access when possible.Evgeniy Stepanov2018-01-231-1/+50
| | | | | | | | | | | | | | | Summary: This adds an -mllvm flag that forces the use of a runtime function call to get the unsafe stack pointer, the same that is currently used on non-x86, non-aarch64 android. The call may be inlined. Reviewers: pcc Subscribers: aemerson, kristof.beyls, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D37405 llvm-svn: 323259
* CodeGen: Fix assertion in ScheduleDAGMILive::scheduleMI due to llvm.dbg.valueYaxun Liu2018-01-231-0/+1
| | | | | | | | | | Fix a bug in ScheduleDAGMILive::scheduleMI which causes BotRPTracker not tracking CurrentBottom in some rare cases involving llvm.dbg.value. This issues causes amdgcn target to assert when compiling some user codes with -g. Differential Revision: https://reviews.llvm.org/D42394 llvm-svn: 323214
* [CGP] Fix the GV handling in complex addressing modeSerguei Katkov2018-01-231-15/+21
| | | | | | | | | | | | | | | If in complex addressing mode the difference is in GV then base reg should not be installed because we plan to use base reg as a merge point of different GVs. This is a fix for PR35980. Reviewers: reames, john.brawn, santosh Reviewed By: john.brawn Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D42230 llvm-svn: 323192
* Introduce the "retpoline" x86 mitigation technique for variant #2 of the ↵Chandler Carruth2018-01-225-0/+230
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | speculative execution vulnerabilities disclosed today, specifically identified by CVE-2017-5715, "Branch Target Injection", and is one of the two halves to Spectre.. Summary: First, we need to explain the core of the vulnerability. Note that this is a very incomplete description, please see the Project Zero blog post for details: https://googleprojectzero.blogspot.com/2018/01/reading-privileged-memory-with-side.html The basis for branch target injection is to direct speculative execution of the processor to some "gadget" of executable code by poisoning the prediction of indirect branches with the address of that gadget. The gadget in turn contains an operation that provides a side channel for reading data. Most commonly, this will look like a load of secret data followed by a branch on the loaded value and then a load of some predictable cache line. The attacker then uses timing of the processors cache to determine which direction the branch took *in the speculative execution*, and in turn what one bit of the loaded value was. Due to the nature of these timing side channels and the branch predictor on Intel processors, this allows an attacker to leak data only accessible to a privileged domain (like the kernel) back into an unprivileged domain. The goal is simple: avoid generating code which contains an indirect branch that could have its prediction poisoned by an attacker. In many cases, the compiler can simply use directed conditional branches and a small search tree. LLVM already has support for lowering switches in this way and the first step of this patch is to disable jump-table lowering of switches and introduce a pass to rewrite explicit indirectbr sequences into a switch over integers. However, there is no fully general alternative to indirect calls. We introduce a new construct we call a "retpoline" to implement indirect calls in a non-speculatable way. It can be thought of loosely as a trampoline for indirect calls which uses the RET instruction on x86. Further, we arrange for a specific call->ret sequence which ensures the processor predicts the return to go to a controlled, known location. The retpoline then "smashes" the return address pushed onto the stack by the call with the desired target of the original indirect call. The result is a predicted return to the next instruction after a call (which can be used to trap speculative execution within an infinite loop) and an actual indirect branch to an arbitrary address. On 64-bit x86 ABIs, this is especially easily done in the compiler by using a guaranteed scratch register to pass the target into this device. For 32-bit ABIs there isn't a guaranteed scratch register and so several different retpoline variants are introduced to use a scratch register if one is available in the calling convention and to otherwise use direct stack push/pop sequences to pass the target address. This "retpoline" mitigation is fully described in the following blog post: https://support.google.com/faqs/answer/7625886 We also support a target feature that disables emission of the retpoline thunk by the compiler to allow for custom thunks if users want them. These are particularly useful in environments like kernels that routinely do hot-patching on boot and want to hot-patch their thunk to different code sequences. They can write this custom thunk and use `-mretpoline-external-thunk` *in addition* to `-mretpoline`. In this case, on x86-64 thu thunk names must be: ``` __llvm_external_retpoline_r11 ``` or on 32-bit: ``` __llvm_external_retpoline_eax __llvm_external_retpoline_ecx __llvm_external_retpoline_edx __llvm_external_retpoline_push ``` And the target of the retpoline is passed in the named register, or in the case of the `push` suffix on the top of the stack via a `pushl` instruction. There is one other important source of indirect branches in x86 ELF binaries: the PLT. These patches also include support for LLD to generate PLT entries that perform a retpoline-style indirection. The only other indirect branches remaining that we are aware of are from precompiled runtimes (such as crt0.o and similar). The ones we have found are not really attackable, and so we have not focused on them here, but eventually these runtimes should also be replicated for retpoline-ed configurations for completeness. For kernels or other freestanding or fully static executables, the compiler switch `-mretpoline` is sufficient to fully mitigate this particular attack. For dynamic executables, you must compile *all* libraries with `-mretpoline` and additionally link the dynamic executable and all shared libraries with LLD and pass `-z retpolineplt` (or use similar functionality from some other linker). We strongly recommend also using `-z now` as non-lazy binding allows the retpoline-mitigated PLT to be substantially smaller. When manually apply similar transformations to `-mretpoline` to the Linux kernel we observed very small performance hits to applications running typical workloads, and relatively minor hits (approximately 2%) even for extremely syscall-heavy applications. This is largely due to the small number of indirect branches that occur in performance sensitive paths of the kernel. When using these patches on statically linked applications, especially C++ applications, you should expect to see a much more dramatic performance hit. For microbenchmarks that are switch, indirect-, or virtual-call heavy we have seen overheads ranging from 10% to 50%. However, real-world workloads exhibit substantially lower performance impact. Notably, techniques such as PGO and ThinLTO dramatically reduce the impact of hot indirect calls (by speculatively promoting them to direct calls) and allow optimized search trees to be used to lower switches. If you need to deploy these techniques in C++ applications, we *strongly* recommend that you ensure all hot call targets are statically linked (avoiding PLT indirection) and use both PGO and ThinLTO. Well tuned servers using all of these techniques saw 5% - 10% overhead from the use of retpoline. We will add detailed documentation covering these components in subsequent patches, but wanted to make the core functionality available as soon as possible. Happy for more code review, but we'd really like to get these patches landed and backported ASAP for obvious reasons. We're planning to backport this to both 6.0 and 5.0 release streams and get a 5.0 release with just this cherry picked ASAP for distros and vendors. This patch is the work of a number of people over the past month: Eric, Reid, Rui, and myself. I'm mailing it out as a single commit due to the time sensitive nature of landing this and the need to backport it. Huge thanks to everyone who helped out here, and everyone at Intel who helped out in discussions about how to craft this. Also, credit goes to Paul Turner (at Google, but not an LLVM contributor) for much of the underlying retpoline design. Reviewers: echristo, rnk, ruiu, craig.topper, DavidKreitzer Subscribers: sanjoy, emaste, mcrosier, mgorny, mehdi_amini, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D41723 llvm-svn: 323155
* Fixing warnings caused by commit 323095Marina Yatsina2018-01-223-10/+10
| | | | | Change-Id: I4e1f81db2f5382a820f4016c23b243e4d5aebf51 llvm-svn: 323114
* Separate LoopTraversal, ReachingDefAnalysis and BreakFalseDeps into their ↵Marina Yatsina2018-01-225-417/+547
| | | | | | | | | | | | | | | | | | own files. This is the one of multiple patches that fix bugzilla https://bugs.llvm.org/show_bug.cgi?id=33869 Most of the patches are intended at refactoring the existent code. Additional relevant reviews: https://reviews.llvm.org/D40330 https://reviews.llvm.org/D40331 https://reviews.llvm.org/D40332 https://reviews.llvm.org/D40334 Differential Revision: https://reviews.llvm.org/D40333 Change-Id: Ie5f8eb34d98cfdfae23a3072eb69b5794f0e2d56 llvm-svn: 323095
* Rename ExecutionDepsFix files to ExecutionDomainFixMarina Yatsina2018-01-222-3/+3
| | | | | | | | | | | | | | | | This is the one of multiple patches that fix bugzilla https://bugs.llvm.org/show_bug.cgi?id=33869 Most of the patches are intended at refactoring the existent code. Additional relevant reviews: https://reviews.llvm.org/D40330 https://reviews.llvm.org/D40331 https://reviews.llvm.org/D40333 https://reviews.llvm.org/D40334 Differential Revision: https://reviews.llvm.org/D40332 Change-Id: I6a048cca7fdafbfc42fb1bac94343e483befded8 llvm-svn: 323094
* ExecutionDepsFix refactoring:Marina Yatsina2018-01-221-53/+67
| | | | | | | | | | | | | | | | | | - clang-format This is the one of multiple patches that fix bugzilla https://bugs.llvm.org/show_bug.cgi?id=33869 Most of the patches are intended at refactoring the existent code. Additional relevant reviews: https://reviews.llvm.org/D40330 https://reviews.llvm.org/D40332 https://reviews.llvm.org/D40333 https://reviews.llvm.org/D40334 Differential Revision: https://reviews.llvm.org/D40331 Change-Id: I131b126af13bc743bc5d69d83699e52b9b720979 llvm-svn: 323093
OpenPOWER on IntegriCloud