summaryrefslogtreecommitdiffstats
path: root/mlir/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
...
* NFC. Move all of the remaining operations left in BuiltinOps to StandardOps. ↵River Riddle2019-03-2912-11/+2
| | | | | | The only thing left in BuiltinOps are the core MLIR types. The standard types can't be moved because they are referenced within the IR directory, e.g. in things like Builder. PiperOrigin-RevId: 236403665
* Use consistent names for dialect op source filesLei Zhang2019-03-2910-10/+10
| | | | | | | | | | | | | | | This CL changes dialect op source files (.h, .cpp, .td) to follow the following convention: <full-dialect-name>/<dialect-namespace>Ops.{h|cpp|td} Builtin and standard dialects are specially treated, though. Both of them do not have dialect namespace; the former is still named as BuiltinOps.* and the latter is named as Ops.*. Purely mechanical. NFC. PiperOrigin-RevId: 236371358
* Loop fusion for input reuse.MLIR Team2019-03-291-43/+415
| | | | | | | | | | | | | *) Breaks fusion pass into multiple sub passes over nodes in data dependence graph: - first pass fuses single-use producers into their unique consumer. - second pass enables fusing for input-reuse by fusing sibling nodes which read from the same memref, but which do not share dependence edges. - third pass fuses remaining producers into their consumers (Note that the sibling fusion pass may have transformed a producer with multiple uses into a single-use producer). *) Fusion for input reuse is enabled by computing a sibling node slice using the load/load accesses to the same memref, and fusion safety is guaranteed by checking that the sibling node memref write region (to a different memref) is preserved. *) Enables output vector and output matrix computations from KFAC patches-second-moment operation to fuse into a single loop nest and reuse input from the image patches operation. *) Adds a generic loop utilitiy for finding all sequential loops in a loop nest. *) Adds and updates unit tests. PiperOrigin-RevId: 236350987
* Provide a Builder::getNamedAttr and ↵River Riddle2019-03-292-6/+4
| | | | | | (Instruction|Function)::setAttr(StringRef, Attribute) to simplify attribute manipulation. PiperOrigin-RevId: 236222504
* Remove PassResult and have the runOnFunction/runOnModule functions return ↵River Riddle2019-03-2918-88/+56
| | | | | | void instead. To signal a pass failure, passes should now invoke the 'signalPassFailure' method. This provides the equivalent functionality when needed, but isn't an intrusive part of the API like PassResult. PiperOrigin-RevId: 236202029
* Change some of the debug messages to use emitError / emitWarning / emitNote ↵Uday Bondhugula2019-03-292-6/+7
| | | | | | - NFC PiperOrigin-RevId: 236169676
* Port all of the existing passes over to the new pass manager infrastructure. ↵River Riddle2019-03-2918-172/+125
| | | | | | This is largely NFC. PiperOrigin-RevId: 235952357
* Temp change in FlatAffineConstraints::getSliceBounds() to deal with TODO inUday Bondhugula2019-03-291-2/+1
| | | | | | | | | | | LoopFusion - getConstDifference in LoopFusion is pending a refactoring to handle bounds with min's and max's; it currently asserts on some useful test cases that we want to experiment with. This CL changes getSliceBounds to be more conservative so as to not trigger the assertion. Filed b/126426796 to track this. PiperOrigin-RevId: 235826538
* Loop fusion comand line options cleanupUday Bondhugula2019-03-291-10/+25
| | | | | | | | - clean up loop fusion CL options for promoting local buffers to fast memory space - add parameters to loop fusion pass instantiation PiperOrigin-RevId: 235813419
* Rewrite the dominance info classes to allow for operating on arbitrary ↵River Riddle2019-03-291-14/+27
| | | | | | control flow within operation regions. The CSE pass is also updated to properly handle nested dominance. PiperOrigin-RevId: 235742627
* Add a stripmineSink and imperfectly nested tiling primitives.Nicolas Vasilache2019-03-291-0/+95
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This CL adds a primitive to perform stripmining of a loop by a given factor and sinking it under multiple target loops. In turn this is used to implement imperfectly nested loop tiling (with interchange) by repeatedly calling the stripmineSink primitive. The API returns the point loops and allows repeated invocations of tiling to achieve declarative, multi-level, imperfectly-nested tiling. Note that this CL is only concerned with the mechanical aspects and does not worry about analysis and legality. The API is demonstrated in an example which creates an EDSC block, emits the corresponding MLIR and applies imperfectly-nested tiling: ```cpp auto block = edsc::block({ For(ArrayRef<edsc::Expr>{i, j}, {zero, zero}, {M, N}, {one, one}, { For(k1, zero, O, one, { C({i, j, k1}) = A({i, j, k1}) + B({i, j, k1}) }), For(k2, zero, O, one, { C({i, j, k2}) = A({i, j, k2}) + B({i, j, k2}) }), }), }); // clang-format on emitter.emitStmts(block.getBody()); auto l_i = emitter.getAffineForOp(i), l_j = emitter.getAffineForOp(j), l_k1 = emitter.getAffineForOp(k1), l_k2 = emitter.getAffineForOp(k2); auto indicesL1 = mlir::tile({l_i, l_j}, {512, 1024}, {l_k1, l_k2}); auto l_ii1 = indicesL1[0][0], l_jj1 = indicesL1[1][0]; mlir::tile({l_jj1, l_ii1}, {32, 16}, l_jj1); ``` The edsc::Expr for the induction variables (i, j, k_1, k_2) provide the programmatic hooks from which tiling can be applied declaratively. PiperOrigin-RevId: 235548228
* Refactor AffineExprFlattener and move FlatAffineConstraints out of IR intoUday Bondhugula2019-03-296-6/+6
| | | | | | | | | | | | | | | | | Analysis - NFC - refactor AffineExprFlattener (-> SimpleAffineExprFlattener) so that it doesn't depend on FlatAffineConstraints, and so that FlatAffineConstraints could be moved out of IR/; the simplification that the IR needs for AffineExpr's doesn't depend on FlatAffineConstraints - have AffineExprFlattener derive from SimpleAffineExprFlattener to use for all Analysis/Transforms purposes; override addLocalFloorDivId in the derived class - turn addAffineForOpDomain into a method on FlatAffineConstraints - turn AffineForOp::getAsValueMap into an AffineValueMap ctor PiperOrigin-RevId: 235283610
* NFC: Make DialectConversion not directly inherit from ModulePass. It is now ↵River Riddle2019-03-291-2/+2
| | | | | | just a utility class that performs dialect conversion on a provided module. PiperOrigin-RevId: 235194067
* Rewrite MLPatternLoweringPass to no longer inherit from FunctionPass and ↵River Riddle2019-03-291-5/+8
| | | | | | just provide a utility function that applies ML patterns. PiperOrigin-RevId: 235194034
* Internal changeMLIR Team2019-03-292-2/+2
| | | | PiperOrigin-RevId: 235191129
* Define a PassID class to use when defining a pass. This allows for the type ↵River Riddle2019-03-2918-54/+18
| | | | | | used for the ID field to be self documenting. It also allows for the compiler to know the set alignment of the ID object, which is useful for storing pointer identifiers within llvm data structures. PiperOrigin-RevId: 235107957
* Print debug message better + switch a dma-generate cl opt to uint64_tUday Bondhugula2019-03-291-6/+6
| | | | PiperOrigin-RevId: 234840316
* Extend/improve getSliceBounds() / complete TODO + update unionBoundingBoxUday Bondhugula2019-03-291-11/+15
| | | | | | | | | - compute slices precisely where the destination iteration depends on multiple source iterations (instead of over-approximating to the whole source loop extent) - update unionBoundingBox to deal with input with non-matching symbols - reenable disabled backend test case PiperOrigin-RevId: 234714069
* NFC: Refactor the files related to passes.River Riddle2019-03-2918-18/+18
| | | | | | | * PassRegistry is split into its own source file. * Pass related files are moved to a new library 'Pass'. PiperOrigin-RevId: 234705771
* DMA placement update - hoist loops invariant DMAsUday Bondhugula2019-03-291-16/+69
| | | | | | | - hoist DMAs past all loops immediately surrounding the region that the latter is invariant on - do this at DMA generation time itself PiperOrigin-RevId: 234628447
* Update pass documentation + improve/fix some commentsUday Bondhugula2019-03-291-3/+9
| | | | | | | - add documentation for passes - improve / fix outdated doc comments PiperOrigin-RevId: 234627076
* Add a generic pattern matcher for matching constant values produced by an ↵River Riddle2019-03-291-9/+4
| | | | | | operation with zero operands and a single result. PiperOrigin-RevId: 234616691
* EDSC: make Expr typed and extensibleAlex Zinenko2019-03-291-9/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Expose the result types of edsc::Expr, which are now stored for all types of Exprs and not only for the variadic ones. Require return types when an Expr is constructed, if it will ever have some. An empty return type list is interpreted as an Expr that does not create a value (e.g. `return` or `store`). Conceptually, all edss::Exprs are now typed, with the type being a (potentially empty) tuple of return types. Unbound expressions and Bindables must now be constructed with a specific type they will take. This makes EDSC less evidently type-polymorphic, but we can still write generic code such as Expr sumOfSquares(Expr lhs, Expr rhs) { return lhs * lhs + rhs * rhs; } and use it to construct different typed expressions as sumOfSquares(Bindable(IndexType::get(ctx)), Bindable(IndexType::get(ctx))); sumOfSquares(Bindable(FloatType::getF32(ctx)), Bindable(FloatType::getF32(ctx))); On the positive side, we get the following. 1. We can now perform type checking when constructing Exprs rather than during MLIR emission. Nevertheless, this is still duplicates the Op::verify() until we can factor out type checking from that. 2. MLIREmitter is significantly simplified. 3. ExprKind enum is only used for actual kinds of expressions. Data structures are converging with AbstractOperation, and the users can now create a VariadicExpr("canonical_op_name", {types}, {exprs}) for any operation, even an unregistered one without having to extend the enum and make pervasive changes to EDSCs. On the negative side, we get the following. 1. Typed bindables are more verbose, even in Python. 2. We lose the ability to do print debugging for higher-level EDSC abstractions that are implemented as multiple MLIR Ops, for example logical disjunction. This is the step 2/n towards making EDSC extensible. *** Move MLIR Op construction from MLIREmitter::emitExpr to Expr::build since Expr now has sufficient information to build itself. This is the step 3/n towards making EDSC extensible. Both of these strive to minimize the amount of irrelevant changes. In particular, this introduces more complex pretty-printing for affine and binary expression to make sure tests continue to pass. It also relies on string comparison to identify specific operations that an Expr produces. PiperOrigin-RevId: 234609882
* EDSC: introduce support for blocksAlex Zinenko2019-03-291-5/+5
| | | | | | | | | | | | | | | | | | | | | | | EDSC currently implement a block as a statement that is itself a list of statements. This suffers from two modeling problems: (1) these blocks are not addressable, i.e. one cannot create an instruction where thus constructed block is a successor; (2) they support block nesting, which is not supported by MLIR blocks. Furthermore, emitting such "compound statement" (misleadingly named `Block` in Python bindings) does not actually produce a new Block in the IR. Implement support for creating actual IR Blocks in EDSC. In particular, define a new StmtBlock EDSC class that is neither an Expr nor a Stmt but contains a list of Stmts. Additionally, StmtBlock may have (early-) typed arguments. These arguments are Bindable expressions that can be used inside the block. Provide two calls in the MLIREmitter, `emitBlock` that actually emits a new block and `emitBlockBody` that only emits the instructions contained in the block without creating a new block. In the latter case, the instructions must not use block arguments. Update Python bindings to make it clear when instruction emission happens without creating a new block. PiperOrigin-RevId: 234556474
* Misc. updates/fixes to analysis utils used for DMA generation; update DMAUday Bondhugula2019-03-293-57/+72
| | | | | | | | | | | | | | | | | | | | | | | | | generation pass to make it drop certain assumptions, complete TODOs. - multiple fixes for getMemoryFootprintBytes - pass loopDepth correctly from getMemoryFootprintBytes() - use union while computing memory footprints - bug fixes for addAffineForOpDomain - take into account loop step - add domains of other loop IVs in turn that might have been used in the bounds - dma-generate: drop assumption of "non-unit stride loops being tile space loops and skipping those and recursing to inner depths"; DMA generation is now purely based on available fast mem capacity and memory footprint's calculated - handle memory region compute failures/bailouts correctly from dma-generate - loop tiling cleanup/NFC - update some debug and error messages to use emitNote/emitError in pipeline-data-transfer pass - NFC PiperOrigin-RevId: 234245969
* Support fusing producer loop nests which write to a memref which is live ↵MLIR Team2019-03-291-21/+96
| | | | | | out, provided that the write region of the consumer loop nest to the same memref is a super set of the producer's write region. PiperOrigin-RevId: 234240958
* LoopFusion: perform a series of loop interchanges to increase the loop depth ↵MLIR Team2019-03-292-0/+175
| | | | | | | | | | | | | | at which slices of producer loop nests can be fused into constumer loop nests. *) Adds utility to LoopUtils to perform loop interchange of two AffineForOps. *) Adds utility to LoopUtils to sink a loop to a specified depth within a loop nest, using a series of loop interchanges. *) Computes dependences between all loads and stores in the loop nest, and classifies each loop as parallel or sequential. *) Computes loop interchange permutation required to sink sequential loops (and raise parallel loop nests) while preserving relative order among them. *) Checks each dependence against the permutation to make sure that dependences would not be violated by the loop interchange transformation. *) Calls loop interchange in LoopFusion pass on consumer loop nests before fusing in producers, sinking loops with loop carried dependences deeper into the consumer loop nest. *) Adds and updates related unit tests. PiperOrigin-RevId: 234158370
* Dialect conversion: decouple function signature conversion from type conversionAlex Zinenko2019-03-291-1/+19
| | | | | | | | | | | | | | | | | | | | | | | | Function types are built-in in MLIR and affect the validity of the IR itself. However, advanced target dialects such as the LLVM IR dialect may include custom function types. Until now, dialect conversion was expecting function types not to be converted to the custom type: although the signatures was allowed to change, the outer type must have been an mlir::FunctionType. This effectively prevented dialect conversion from creating instructions that operate on values of the custom function type. Dissociate function signature conversion from general type conversion. Function signature conversion must still produce an mlir::FunctionType and is used in places where built-in types are required to make IR valid. General type conversion is used for SSA values, including function and block arguments and function results. Exercise this behavior in the LLVM IR dialect conversion by converting function types to LLVM IR function pointer types. The pointer to a function is chosen to provide consistent lowering of higher-order functions: while it is possible to have a value of function type, it is not possible to create a function type accepting a returning another function type. PiperOrigin-RevId: 234124494
* Add -tile-sizes command line option for loop tiling; clean up cl options forUday Bondhugula2019-03-293-31/+39
| | | | | | | | | | | | for dma-generate, loop-unroll. - add -tile-sizes command line option for loop tiling to specify different tile sizes for loops in a band - clean up command line options for loop-unroll, dma-generate (remove cl::hidden) PiperOrigin-RevId: 234006232
* Generate dealloc's for alloc's of pipeline-data-transferUday Bondhugula2019-03-291-2/+13
| | | | | | | | | | | | - for the DMA transfers being pipelined through double buffering, generate deallocs for the double buffers being alloc'ed This change is along the lines of cl/233502632. We initially wanted to experiment with scoped allocation - so the deallocation's were usually not necessary; however, they are needed even with scoped allocations in some situations - for eg. when the enclosing loop gets unrolled. The dealloc serves as an end of lifetime marker. PiperOrigin-RevId: 233653463
* Make IndexType a standard type instead of a builtin. This also cleans up ↵River Riddle2019-03-292-4/+5
| | | | | | some unnecessary factory methods on the Type class. PiperOrigin-RevId: 233640730
* EDSC: move Expr and Stmt construction operators to a namespaceAlex Zinenko2019-03-291-0/+1
| | | | | | | | | | | | | | | | | | In the current state, edsc::Expr and edsc::Stmt overload operators to construct other Exprs and Stmts. This includes some unconventional overloads of the `operator==` to create a comparison expression and of the `operator!` to create a negation expression. This situation could lead to unpleasant surprises where the code does not behave like expected. Make all Expr and Stmt construction operators free functions and move them to the `edsc::op` namespace. Callers willing to use these operators must explicitly include them with the `using` declaration. This can be done in some local scope. Additionally, we currently emit signed comparisons for order-comparison operators. With namespaces, we can later introduce two sets of operators in different namespace, e.g. `edsc::op::sign` and `edsc::op::unsign` to clearly state which kind of comparison is implied. PiperOrigin-RevId: 233578674
* Generate dealloc's for the alloc's of dma-generate.Uday Bondhugula2019-03-293-13/+36
| | | | | | | | | | | | | | | | | - for the DMA buffers being allocated (and their tags), generate corresponding deallocs - minor related update to replaceAllMemRefUsesWith and PipelineDataTransfer pass Code generation for DMA transfers was being done with the initial simplifying assumption that the alloc's would map to scoped allocations, and so no deallocations would be necessary. Drop this assumption to generalize. Note that even with scoped allocations, unrolling loops that have scoped allocations could create a series of allocations and exhaustion of fast memory. Having a end of lifetime marker like a dealloc in fact allows creating new scopes if necessary when lowering to a backend and still utilize scoped allocation. DMA buffers created by -dma-generate are guaranteed to have either non-overlapping lifetimes or nested lifetimes. PiperOrigin-RevId: 233502632
* Remove the restriction that only registered terminator operations may ↵River Riddle2019-03-292-2/+2
| | | | | | terminate a block and have block operands. This allows for any operation to hold block operands. It also introduces the notion that unregistered operations may terminate a block. As such, the 'isTerminator' api on Instruction has been split into 'isKnownTerminator' and 'isKnownNonTerminator'. PiperOrigin-RevId: 233076831
* Automated rollback of changelist 232728977.Uday Bondhugula2019-03-293-19/+19
| | | | PiperOrigin-RevId: 232944889
* Modify the canonicalizations of select and muli to use the fold hook.River Riddle2019-03-291-17/+33
| | | | | | This also extends the greedy pattern rewrite driver to add the operands of folded operations back to the worklist. PiperOrigin-RevId: 232878959
* Automated rollback of changelist 232717775.Uday Bondhugula2019-03-2913-80/+76
| | | | PiperOrigin-RevId: 232807986
* When canonicalizing only erase the operation after calling the 'fold' hook ↵River Riddle2019-03-291-2/+2
| | | | | | if replacement results were supplied. This fixes a bug where the operation would always get erased, even if it was modified in place. PiperOrigin-RevId: 232757964
* Rename the 'if' operation in the AffineOps dialect to 'affine.if' and namespaceRiver Riddle2019-03-293-20/+20
| | | | | | the AffineOps dialect with 'affine'. PiperOrigin-RevId: 232728977
* NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for'. ↵River Riddle2019-03-2913-80/+85
| | | | | | The is the second step to adding a namespace to the AffineOps dialect. PiperOrigin-RevId: 232717775
* NFC: Rename affine_apply to affine.apply. This is the first step to adding a ↵River Riddle2019-03-297-18/+18
| | | | | | namespace to the affine dialect. PiperOrigin-RevId: 232707862
* Adds the ability to compute the MemRefRegion of a sliced loop nest. Utilizes ↵MLIR Team2019-03-291-11/+27
| | | | | | | | | this feature during loop fusion cost computation, to compute what the write region of a fusion candidate loop nest slice would be (without having to materialize the slice or change the IR). *) Adds parameter to public API of MemRefRegion::compute for passing in the slice loop bounds to compute the memref region of the loop nest slice. *) Exposes public method MemRefRegion::getRegionSize for computing the size of the memref region in bytes. PiperOrigin-RevId: 232706165
* Move the AffineFor loop bound folding to a canonicalization pattern on the ↵River Riddle2019-03-292-59/+0
| | | | | | AffineForOp. PiperOrigin-RevId: 232610715
* Refactor the affine analysis by moving some functionality to IR and some to ↵River Riddle2019-03-296-101/+5
| | | | | | | | | | | | | | AffineOps. This is important for allowing the affine dialect to define canonicalizations directly on the operations instead of relying on transformation passes, e.g. ComposeAffineMaps. A summary of the refactoring: * AffineStructures has moved to IR. * simplifyAffineExpr/simplifyAffineMap/getFlattenedAffineExpr have moved to IR. * makeComposedAffineApply/fullyComposeAffineMapAndOperands have moved to AffineOps. * ComposeAffineMaps is replaced by AffineApplyOp::canonicalize and deleted. PiperOrigin-RevId: 232586468
* Loop fusion improvements:MLIR Team2019-03-291-7/+28
| | | | | | | *) After a private memref buffer is created for a fused loop nest, dependences on the old memref are reduced, which can open up fusion opportunities. In these cases, users of the old memref are added back to the worklist to be reconsidered for fusion. *) Fixed a bug in fusion insertion point dependence check where the memref being privatized was being skipped from the check. PiperOrigin-RevId: 232477853
* Remove stray debug output - NFCUday Bondhugula2019-03-291-1/+0
| | | | PiperOrigin-RevId: 232390076
* Remove InstWalker and move all instruction walking to the api facilities on ↵River Riddle2019-03-299-128/+88
| | | | | | Function/Block/Instruction. PiperOrigin-RevId: 232388113
* NFC: Move AffineApplyOp to the AffineOps dialect. This also moves the ↵River Riddle2019-03-293-1/+3
| | | | | | isValidDim/isValidSymbol methods from Value to the AffineOps dialect. PiperOrigin-RevId: 232386632
* Refactor common code getting memref access in getMemRefRegion - NFCUday Bondhugula2019-03-294-9/+13
| | | | | | | | | | | | | | | - use getAccessMap() instead of repeating it - fold getMemRefRegion into MemRefRegion ctor (more natural, avoid heap allocation and unique_ptr where possible) - change extractForInductionVars - MutableArrayRef -> ArrayRef for the arguments. Since the method is just returning copies of 'Value *', the client can't mutate the pointers themselves; it's fine to mutate the 'Value''s themselves, but that doesn't mutate the pointers to those. - change the way extractForInductionVars returns (see b/123437690) PiperOrigin-RevId: 232359277
* Remove remaining usages of OperationInst in lib/Transforms.River Riddle2019-03-2920-317/+251
| | | | PiperOrigin-RevId: 232323671
OpenPOWER on IntegriCloud