summaryrefslogtreecommitdiffstats
path: root/mlir/lib/IR
Commit message (Collapse)AuthorAgeFilesLines
...
* Add OpTraits for operand types: IntegerLike and SameType.Alex Zinenko2019-03-291-12/+34
| | | | | | | | | | | | | | Introduce new OpTraits verifying relation between operands of an Operation, similarly to its results. Arithmetic operations are defined separately for integer and floating point types. While we are currently leveraging the equality of result and operand types to make sure the right arithmetic operations are used for the right types, we may eventually want to verify operand types directly. Furthermore, for upcoming comparison operations, the type of the result differs from those of the operands so we need to verify the operand types directly. Similarly, we will want to restrict comparisons (and potentially binary arithmetic operations) to operands of the same type. PiperOrigin-RevId: 220365629
* Uniformize MemRefType well-formedness checks.Alex Zinenko2019-03-291-8/+60
| | | | | | | | | | | | | | | | | | | | | | | Introduce a new public static member function, MemRefType::getChecked, intended for the users that want detailed error messages to be emitted during MemRefType construction and can gracefully handle these errors. This function takes a Location of the "MemRef" token if known. The parser is one user of getChecked that has location information, it outputs errors as compiler diagnostics. Other users may pass in an instance of UnknownLoc and still have error messages emitted. Compiler-internal users not expecting the MemRefType construction to fail should call MemRefType::get, which now aborts on failure with a generic message. Both "getChecked" and "get" call to a static free function that does actual construction with well-formedness checks, optionally emits errors and returns nullptr on failure. The location information passed to getChecked has voluntarily coarse precision. The error messages are intended for compiler engineers and do not justify heavier API than a single location. The text of the messages can be written so that it pinpoints the actual location of the error within a MemRef declaration. PiperOrigin-RevId: 219765902
* Adds a dependence check to test whether two accesses to the same memref ↵MLIR Team2019-03-291-0/+9
| | | | | | | | | | | | access the same element. - Builds access functions and iterations domains for each access. - Builds dependence polyhedron constraint system which has equality constraints for equated access functions and inequality constraints for iteration domain loop bounds. - Runs elimination on the dependence polyhedron to test if no dependence exists between the accesses. - Adds a trivial LoopFusion transformation pass with a simple test policy to test dependence between accesses to the same memref in adjacent loops. - The LoopFusion pass will be extended in subsequent CLs. PiperOrigin-RevId: 219630898
* Implement value type abstraction for types.River Riddle2019-03-2913-270/+431
| | | | | | This is done by changing Type to be a POD interface around an underlying pointer storage and adding in-class support for isa/dyn_cast/cast. PiperOrigin-RevId: 219372163
* [MLIR] Implement 1-D vectorization for fastest varying load/storesNicolas Vasilache2019-03-291-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This CL is a first in a series that implements early vectorization of increasingly complex patterns. In particular, early vectorization will support arbitrary loop nesting patterns (both perfectly and imperfectly nested), at arbitrary depths in the loop tree. This first CL builds the minimal support for applying 1-D patterns. It relies on an unaligned load/store op abstraction that can be inplemented differently on different HW. Future CLs will support higher dimensional patterns, but 1-D patterns already exhibit interesting properties. In particular, we want to separate pattern matching (i.e. legality both structural and dependency analysis based), from profitability analysis, from application of the transformation. As a consequence patterns may intersect and we need to verify that a pattern can still apply by the time we get to applying it. A non-greedy analysis on profitability that takes into account pattern intersection is left for future work. Additionally the CL makes the following cleanups: 1. the matches method now returns a value, not a reference; 2. added comments about the MLFunctionMatcher and MLFunctionMatches usage by value; 3. added size and empty methods to matches; 4. added a negative vectorization test with a conditional, this exhibited a but in the iterators. Iterators now return nullptr if the underlying storage is nullpt. PiperOrigin-RevId: 219299489
* Drop unbounded identity map from MemRef affine map composition.Alex Zinenko2019-03-291-6/+9
| | | | | | | | | | | | | | | | Unbounded identity maps do not affect the accesses through MemRefs in any way. A previous CL dropped such maps only if they were alone in the composition. Go further and drop such maps everywhere they appear in the composition. Update the parser test to check for unique'd hoisted map to be present but without assuming any particular order. Because some of the hoisted identity maps still apear due to the nested "for" statements, we need to check for them. However, they no longer appear above the non-identity maps because they are no longer necessary for the extfunc memref declarations that are textually first in the test file. This order may change further as map simplification is improved, there is no reason to assume a particular order. PiperOrigin-RevId: 219287280
* Use matcher sugars for cannonicalization pattern matchingLei Zhang2019-03-291-0/+6
| | | | | | | | | - Added a mechanism for specifying pattern matching more concisely like LLVM. - Added support for canonicalization of addi/muli over vector/tensor splat - Added ValueType to Attribute class hierarchy - Allowed creating constant splat PiperOrigin-RevId: 219149621
* [trivial] fix MLIRContext::registerDiagnosticHandler documentationAlex Zinenko2019-03-291-3/+3
| | | | | | | | | The documentation for MLIRContext::registerDiagnosticHandler describing the arguments of the diagnostic handler is inconsistent with the code. It also mentions LLVM context rather than MLIR context, likely a typo. Fix both issues. PiperOrigin-RevId: 219120954
* Add support for walking the use list of an SSAValue and converting owners toChris Lattner2019-03-291-0/+32
| | | | | | | | Operation*'s, simplifying some code in GreedyPatternRewriteDriver.cpp. Also add print/dump methods on Operation. PiperOrigin-RevId: 219045764
* Introduce a common base class (IROperandOwner) between Instruction andChris Lattner2019-03-293-19/+25
| | | | | | | | | | | | | Statement, which paves the way to make SSAValue's have a useful owner available, which will allow subsequent patches to improve their use/def chains. While I'm poking at this, shrink sizeof(Instruction) and sizeof(Statement) by a word by packing the kind and location together into a single PointerIntPair. NFC. PiperOrigin-RevId: 218959651
* Drop trivial identity affine mappings in MemRef construction.Alex Zinenko2019-03-291-0/+8
| | | | | | | | | | | | | | | | | | | As per MLIR spec, the absence of affine maps in MemRef type is interpreted as an implicit identity affine map. Therefore, MemRef types declared with explicit or implicit identity map should be considered equal at the MemRefType level. During MemRefType construction, drop trivial identity affine map compositions. A trivial identity composition consists of a single unbounded identity map. It is unclear whether affine maps should be composed in-place to a single map during MemRef type construction, so non-trivial compositions that could have been simplified to an identity are NOT removed. We chose to drop the trivial identity map rather than inject it in places that assume its present implicitly because it makes the code simpler by reducing boilerplate; identity mappings are obvious defaults. Update tests that were checking for the presence of trivial identity map compositions in the outputs. PiperOrigin-RevId: 218862454
* Move AffineMap.h/IntegerSet.h from Attributes.h to AttributeDetail.h where ↵River Riddle2019-03-291-0/+3
| | | | | | they belong. PiperOrigin-RevId: 218806426
* Introduce integer set attributeUday Bondhugula2019-03-295-0/+36
| | | | | | | - add IntegerSetAttr to Attributes; add parsing and other support for it (builder, etc.). PiperOrigin-RevId: 218804579
* Change sigil for integer set: @@ -> #Uday Bondhugula2019-03-291-1/+1
| | | | PiperOrigin-RevId: 218786684
* Refactor all of the canonicalization patterns out of the Canonicalize pass, andChris Lattner2019-03-292-0/+228
| | | | | | | | | | make operations provide a list of canonicalizations that can be applied to them. This allows canonicalization to be general to any IR definition. As part of this, sink PatternMatch.h/cpp down to the IR library to fix a layering problem. PiperOrigin-RevId: 218773981
* Implement value type abstraction for attributes.River Riddle2019-03-298-293/+576
| | | | | | This is done by changing Attribute to be a POD interface around an underlying pointer storage and adding in-class support for isa/dyn_cast/cast. PiperOrigin-RevId: 218764173
* Introduce Fourier-Motzkin variable elimination + other cleanup/supportUday Bondhugula2019-03-296-20/+72
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Introduce Fourier-Motzkin variable elimination to eliminate a dimension from a system of linear equalities/inequalities. Update isEmpty to use this. Since FM is only exact on rational/real spaces, an emptiness check based on this is guaranteed to be exact whenever it says the underlying set is empty; if it says, it's not empty, there may still be no integer points in it. Also, supports a version that computes "dark shadows". - Test this by checking for "always false" conditionals in if statements. - Unique IntegerSet's that are small (few constraints, few variables). This basically means the canonical empty set and other small sets that are likely commonly used get uniqued; allows checking for the canonical empty set by pointer. IntegerSet::kUniquingThreshold gives the threshold constraint size for uniqui'ing. - rename simplify-affine-expr -> simplify-affine-structures Other cleanup - IntegerSet::numConstraints, AffineMap::numResults are no longer needed; remove them. - add copy assignment operators for AffineMap, IntegerSet. - rename Invalid() -> Null() on AffineExpr, AffineMap, IntegerSet - Misc cleanup for FlatAffineConstraints API PiperOrigin-RevId: 218690456
* Adds Gaussian Elimination to FlatAffineConstraints.MLIR Team2019-03-291-0/+16
| | | | | | | | - Adds FlatAffineConstraints::isEmpty method to test if there are no solutions to the system. - Adds GCD test check if equality constraints have no solution. - Adds unit test cases. PiperOrigin-RevId: 218546319
* Teach canonicalize pass to unique and hoist constants to the entry block. ThisChris Lattner2019-03-292-1/+41
| | | | | | | | | is a straight-forward change, but required adding missing moveBefore() methods on operations (requiring moving some traits around to make C++ happy). This also fixes a constness issue with the getBlock/getFunction() methods on Instruction, and adds a missing getFunction() method on MLFuncBuilder. PiperOrigin-RevId: 218523905
* Add support to opaque elements attributesFeng Liu2019-03-293-2/+57
| | | | | | | | | | | | | For some of the constant vector / tesor, if the compiler doesn't need to interpret their elements content, they can be stored in this class to save the serialize / deserialize cost. syntax: `opaque<` tensor-type `,` opaque-string `>` opaque-string ::= `0x` [0-9a-fA-F]* PiperOrigin-RevId: 218399426
* Move the ReturnOp type checks to ReturnOp::verify.Alex Zinenko2019-03-291-1/+15
| | | | | | | | | | | | | | | | This was left as a TODO in the code. Move the type verification from MLFuncVerifier::verifyReturn to ReturnOp::verify. Since the return operation can only appear as the last statement of an MLFunction, i.e. where the surrounding block is the function itself, it is easy to access the function descriptor (ReturnOp::verify already relies on this). From the function descriptor, one can easily access the type information. Note that this slightly modifies the error message due to the use of emitOpError instead of a plain emitError. Drop the obsolete TODO comment in MLFunction::verify about checking that "return" only appears as the last operation of an MLFunction since ReturnOp::verify explicitly checks for that. PiperOrigin-RevId: 218347843
* Add a pattern (x+0) -> x, generalize Canonicalize to CFGFunc's, address a ↵Chris Lattner2019-03-291-0/+8
| | | | | | | | few TODOs, and add some casting support to Operation. PiperOrigin-RevId: 218219340
* Random cleanups:Chris Lattner2019-03-292-1/+3
| | | | | | | | | | - Change AllocOp to have a getType() that always returns a MemRefType, since that is what it requires. - Rename StandardOps/StandardOpRegistration.cpp -> StandardOps/OpRegistration.cpp to align with other op sets. - Add AffineMap::getContext() helper and use it in the asmprinter. PiperOrigin-RevId: 218205527
* introduce a memref_cast operation, refactoring common code between it andChris Lattner2019-03-291-1/+25
| | | | | | shape_cast into a common CastOp class. PiperOrigin-RevId: 218175818
* Introduce a new Operation::erase helper to generalize some code inChris Lattner2019-03-294-4/+11
| | | | | | | the pattern matcher / canonicalizer, and rename existing eraseFromBlock methods to align with it. PiperOrigin-RevId: 218104455
* Replace the "OperationSet" abstraction with a new Dialect abstraction. This isChris Lattner2019-03-296-110/+116
| | | | | | | | | a step forward because now every AbstractOperation knows which Dialect it is associated with, enabling things in the future like "constant folding hooks" which will be important for layering. This is also a bit nicer on the registration side of things. PiperOrigin-RevId: 218104230
* Use APFloat for FloatAttributeFeng Liu2019-03-294-34/+55
| | | | | | | | | | | | We should be able to represent arbitrary precision Float-point values inside the IR, so compiler optimizations, such as constant folding can be done independently on the compiling platform. This CL also added a new field, AttrValueGetter, to the Attr class definition for TableGen. This field is used to customize which mlir::Attr getter method to get the defined PrimitiveType. PiperOrigin-RevId: 218034983
* Loop bound constant folding: follow-up / address comments from cl/215997346Uday Bondhugula2019-03-291-45/+0
| | | | | | | - create a single function to fold both bounds - move bound constant folding into transforms PiperOrigin-RevId: 217954701
* Rename Operation::getAs to Operation::dyn_castFeng Liu2019-03-293-27/+27
| | | | | | | | | Also rename Operation::is to Operation::isa Introduce Operation::cast All of these are for consistency with global dyn_cast/cast/isa operators. PiperOrigin-RevId: 217878786
* Touch an unused variable.MLIR Team2019-03-291-0/+1
| | | | PiperOrigin-RevId: 217861580
* Fix AffineExpr printing bug: paren ellision b/117887365.Uday Bondhugula2019-03-291-1/+5
| | | | PiperOrigin-RevId: 217803621
* Add support to constant sparse tensor / vector attributeFeng Liu2019-03-293-1/+42
| | | | | | | | | | | | | | | | | | | | | | | | | The SparseElementsAttr uses (COO) Coordinate List encoding to represents a sparse tensor / vector. Specifically, the coordinates and values are stored as two dense elements attributes. The first dense elements attribute is a 2-D attribute with shape [N, ndims], which contains the indices of the elements with nonzero values in the constant vector/tensor. The second elements attribute is a 1-D attribute list with shape [N], which supplies the values for each element in the first elements attribute. ndims is the rank of the vector/tensor and N is the total nonzero elements. The syntax is: `sparse<` (tensor-type | vector-type)`, ` indices-attribute-list, values-attribute-list `>` Example: a sparse tensor sparse<vector<3x4xi32>, [[0, 0], [1, 2]], [1, 2]> represents the dense tensor [[1, 0, 0, 0] [0, 0, 2, 0] [0, 0, 0, 0]] PiperOrigin-RevId: 217764319
* Add support to constant dense vector/tensor attribute.Feng Liu2019-03-294-13/+270
| | | | | | | | | | | | | | | | | | | The syntax of dense vecor/tensor attribute value is `dense<` (tensor-type | vector-type)`,` attribute-list`>` and attribute-list ::= `[` attribute-list (`, ` attribute-list)* `]`. The construction of the dense vector/tensor attribute takes a vector/tensor type and a character array as arguments. The size of the input array should be larger than the size specified by the type argument. It also assumes the elements of the vector or tensor have been trunked to the data type sizes in the input character array, so it extends the trunked data to 64 bits when it is retrieved. PiperOrigin-RevId: 217762811
* Generalize / improve DMA transfer overlap; nested and multiple DMA support; ↵Uday Bondhugula2019-03-291-0/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | resolve multiple TODOs. - replace the fake test pass (that worked on just the first loop in the MLFunction) to perform DMA pipelining on all suitable loops. - nested DMAs work now (DMAs in an outer loop, more DMAs in nested inner loops) - fix bugs / assumptions: correctly copy memory space and elemental type of source memref for double buffering. - correctly identify matching start/finish statements, handle multiple DMAs per loop. - introduce dominates/properlyDominates utitilies for MLFunction statements. - move checkDominancePreservationOnShifts to LoopAnalysis.h; rename it getShiftValidity - refactor getContainingStmtPos -> findAncestorStmtInBlock - move into Analysis/Utils.h; has two users. - other improvements / cleanup for related API/utilities - add size argument to dma_wait - for nested DMAs or in general, it makes it easy to obtain the size to use when lowering the dma_wait since we wouldn't want to identify the matching dma_start, and more importantly, in general/in the future, there may not always be a dma_start dominating the dma_wait. - add debug information in the pass PiperOrigin-RevId: 217734892
* [MLIR] Basic infrastructure for vectorization testNicolas Vasilache2019-03-291-0/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This CL implements a very simple loop vectorization **test** and the basic infrastructure to support it. The test simply consists in: 1. matching the loops in the MLFunction and all the Load/Store operations nested under the loop; 2. testing whether all the Load/Store are contiguous along the innermost memory dimension along that particular loop. If any reference is non-contiguous (i.e. the ForStmt SSAValue appears in the expression), then the loop is not-vectorizable. The simple test above can gradually be extended with more interesting behaviors to account for the fact that a layout permutation may exist that enables contiguity etc. All these will come in due time but it is worthwhile noting that the test already supports detection of outer-vetorizable loops. In implementing this test, I also added a recursive MLFunctionMatcher and some sugar that can capture patterns such as `auto gemmLike = Doall(Doall(Red(LoadStore())))` and allows iterating on the matched IR structures. For now it just uses in order traversal but post-order DFS will be useful in the future once IR rewrites start occuring. One may note that the memory management design decision follows a different pattern from MLIR. After evaluating different designs and how they quickly increase cognitive overhead, I decided to opt for the simplest solution in my view: a class-wide (threadsafe) RAII context. This way, a pass that needs MLFunctionMatcher can just have its own locally scoped BumpPtrAllocator and everything is cleaned up when the pass is destroyed. If passes are expected to have a longer lifetime, then the contexts can easily be scoped inside the runOnMLFunction call and storage lifetime reduced. Lastly, whatever the scope of threading (module, function, pass), this is expected to also be future-proof wrt concurrency (but this is a detail atm). PiperOrigin-RevId: 217622889
* Add constant folding and binary operator reassociation to the canonicalizeChris Lattner2019-03-291-0/+8
| | | | | | | pass, build up the worklist infra in anticipation of improving the pattern matcher to match more than one node. PiperOrigin-RevId: 217330579
* Implement a super sketched out pattern match/rewrite framework and a sketchedChris Lattner2019-03-291-0/+8
| | | | | | | | | | | out canonicalization pass to drive it, and a simple (x-x) === 0 pattern match as a test case. There is a tremendous number of improvements that need to land, and the matcher/rewriter and patterns will be split out of this file, but this is a starting point. PiperOrigin-RevId: 216788604
* Add MLFunction::walk/walkPostOrder methods for doing a simple traversal ofChris Lattner2019-03-291-0/+26
| | | | | | operations. This is a simplified form for the existing walker API. PiperOrigin-RevId: 216754991
* Split BuiltinOps out of StandardOps.Jacques Pienaar2019-03-296-1159/+369
| | | | | | | | * Move Return, Constant and AffineApply out into BuiltinOps; * BuiltinOps are always registered, while StandardOps follow the same dynamic registration; * Kept isValidX in MLValue as we don't have a verify on AffineMap so need to keep it callable from Parser (I wanted to move it to be called in verify instead); PiperOrigin-RevId: 216592527
* Simplify simplify functions as follow up on previous CL.Jacques Pienaar2019-03-291-28/+30
| | | | | | Addressing comment from post submit + simplifying the logic. PiperOrigin-RevId: 216560688
* Only simplify floor div, ceil div or mod if the rhs constant >= 1.Jacques Pienaar2019-03-291-3/+3
| | | | | | Else we hit asserts in MathExtras. PiperOrigin-RevId: 216553595
* [MLIR] IntegerSet value typeNicolas Vasilache2019-03-296-48/+115
| | | | | | | | | | | | | | | This CL applies the same pattern as AffineMap to IntegerSet: a simple struct that acts as the storage is allocated in the bump pointer. The IntegerSet is immutable and accessed everywhere by value. Note that unlike AffineMap, it is not possible to remove the MLIRContext parameter when constructing an IntegerSet for now. One possible way to achieve this would be to add an enum to distinguish between the mathematically empty set, the universe set and other sets. This is left for future discussion. PiperOrigin-RevId: 216545361
* Add support to constant splat vector/tensor attribute.Feng Liu2019-03-294-5/+53
| | | | | | | | | This attribute represents a reference to a splat vector or tensor, where all the elements have the same value. The syntax of the attribute is: `splat<` (tensor-type | vector-type)`,` attribute-value `>` PiperOrigin-RevId: 216537997
* Change the representation of an operation name to be either anChris Lattner2019-03-296-15/+42
| | | | | | | | | AbstractOperation* or an Identifier. This makes it possible to get to stuff in AbstractOperation faster than going through a hash table lookup. This makes constant folding a bit faster now, but will become more important with subsequent changes. PiperOrigin-RevId: 216476772
* Support `getShape`, `hasStaticShape` and `getDimSize` methods for all the ↵Feng Liu2019-03-294-10/+23
| | | | | | Vector and Tensor Types. PiperOrigin-RevId: 216447553
* [MLIR] AffineMap value typeNicolas Vasilache2019-03-298-169/+226
| | | | | | | | This CL applies the same pattern as AffineExpr to AffineMap: a simple struct that acts as the storage is allocated in the bump pointer. The AffineMap is immutable and accessed everywhere by value. PiperOrigin-RevId: 216445930
* Add target independent standard DMA ops: dma.start, dma.waitUday Bondhugula2019-03-292-6/+136
| | | | | | | | | | | Add target independent standard DMA ops: dma.start, dma.wait. Update pipeline data transfer to use these to detect DMA ops. While on this - return failure from mlir-opt::performActions if a pass generates invalid output - improve error message for verify 'n' operand traits PiperOrigin-RevId: 216429885
* [MLIR] Sketch AffineExpr value typeNicolas Vasilache2019-03-296-284/+278
| | | | | | | | | | | | | | | | This CL sketches what it takes for AffineExpr to fully have by-value semantics and not be a not-so-smart pointer anymore. This essentially makes the underyling class a simple storage struct and implements the operations on the value type directly. Since there is no forwarding of operations anymore, we can full isolate the storage class and make a hard visibility barrier by moving detail::AffineExpr into AffineExprDetail.h. AffineExprDetail.h is only included where storage-related information is needed. PiperOrigin-RevId: 216385459
* Address comments from previous CL/216216446MLIR Team2019-03-291-1/+1
| | | | PiperOrigin-RevId: 216298139
* [MLIR] AffineExpr final cleanupsNicolas Vasilache2019-03-296-225/+217
| | | | | | | | | | | This CL: 1. performs the global codemod AffineXExpr->AffineXExprClass and AffineXExprRef -> AffineXExpr; 2. simplifies function calls by removing the redundant MLIRContext parameter; 3. adds missing binary operator versions of scalar op AffineExpr where it makes sense. PiperOrigin-RevId: 216242674
OpenPOWER on IntegriCloud