summaryrefslogtreecommitdiffstats
path: root/mlir/lib/Analysis/VectorAnalysis.cpp
Commit message (Collapse)AuthorAgeFilesLines
* [mlir] NFC: Remove Value::operator* and Value::operator-> now that Value is ↵River Riddle2020-01-111-1/+1
| | | | | | | | | | properly value-typed. Summary: These were temporary methods used to simplify the transition. Reviewed By: antiagainst Differential Revision: https://reviews.llvm.org/D72548
* NFC: Replace ValuePtr with Value and remove it now that Value is value-typed.River Riddle2019-12-231-2/+2
| | | | | | ValuePtr was a temporary typedef during the transition to a value-typed Value. PiperOrigin-RevId: 286945714
* Adjust License.txt file to use the LLVM licenseMehdi Amini2019-12-231-13/+4
| | | | PiperOrigin-RevId: 286906740
* NFC: Introduce new ValuePtr/ValueRef typedefs to simplify the transition to ↵River Riddle2019-12-221-2/+2
| | | | | | | | | | Value being value-typed. This is an initial step to refactoring the representation of OpResult as proposed in: https://groups.google.com/a/tensorflow.org/g/mlir/c/XXzzKhqqF_0/m/v6bKb08WCgAJ This change will make it much simpler to incrementally transition all of the existing code to use value-typed semantics. PiperOrigin-RevId: 286844725
* Refactor dependencies to expose Vector transformations as patterns - NFCNicolas Vasilache2019-12-031-1/+1
| | | | | | | | | This CL refactors some of the MLIR vector dependencies to allow decoupling VectorOps, vector analysis, vector transformations and vector conversions from each other. This makes the system more modular and allows extracting VectorToVector into VectorTransforms that do not depend on vector conversions. This refactoring exhibited a bunch of cyclic library dependencies that have been cleaned up. PiperOrigin-RevId: 283660308
* Unify vector op names with other dialects.Jean-Michel Gorius2019-11-221-2/+2
| | | | | | | | | | Change vector op names from VectorFooOp to Vector_FooOp and from vector::VectorFooOp to vector::FooOp. Closes tensorflow/mlir#257 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/257 from Kayjukh:master dfc3a0e04114885aaec8740d5951d6984d6e1577 PiperOrigin-RevId: 281967461
* Implement unrolling of vector ops to finer-grained vector ops as a pattern.Nicolas Vasilache2019-11-201-7/+7
| | | | | | | | | This CL uses the pattern rewrite infrastructure to implement a simple VectorOps -> VectorOps legalization strategy to unroll coarse-grained vector operations into finer grained ones. The transformation is written using local pattern rewrites to allow composition with other rewrites. It proceeds by iteratively introducing fake cast ops and cleaning canonicalizing or lowering them away where appropriate. This is an example of writing transformations as compositions of local pattern rewrites that should enable us to make them significantly more declarative. PiperOrigin-RevId: 281555100
* Move VectorOps to Tablegen - (almost) NFCNicolas Vasilache2019-11-141-1/+1
| | | | | | | | | | | | | | | | This CL moves VectorOps to Tablegen and cleans up the implementation. This is almost NFC but 2 changes occur: 1. an interface change occurs in the padding value specification in vector_transfer_read: the value becomes non-optional. As a shortcut we currently use %f0 for all paddings. This should become an OpInterface for vectorization in the future. 2. the return type of vector.type_cast is trivial and simplified to `memref<vector<...>>` Relevant roundtrip and invalid tests that used to sit in core are moved to the vector dialect. The op documentation is moved to the .td file. PiperOrigin-RevId: 280430869
* Fix minor spelling tweaks (NFC)Kazuaki Ishizaki2019-10-201-1/+1
| | | | | | Closes tensorflow/mlir#177 PiperOrigin-RevId: 275692653
* NFC: Move AffineOps dialect to the Dialect sub-directory.River Riddle2019-08-201-1/+1
| | | | PiperOrigin-RevId: 264482571
* Move Linalg and VectorOps dialects to the Dialect subdir - NFCNicolas Vasilache2019-08-191-1/+1
| | | | PiperOrigin-RevId: 264277760
* NFC: Move LLVMIR, SDBM, and StandardOps to the Dialect/ directory.River Riddle2019-08-191-1/+1
| | | | PiperOrigin-RevId: 264193915
* Add a higher-order vector.extractelement operation in MLIRNicolas Vasilache2019-08-091-2/+2
| | | | | | | | This CL is step 2/n towards building a simple, programmable and portable vector abstraction in MLIR that can go all the way down to generating assembly vector code via LLVM's opt and llc tools. This CL adds the vector.extractelement operation to the MLIR vector dialect as well as the appropriate roundtrip test. Lowering to LLVM will occur in the following CL. PiperOrigin-RevId: 262545089
* Globally change load/store/dma_start/dma_wait operations over to ↵Andy Davis2019-07-031-10/+5
| | | | | | | | | | | affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
* Remove "size" property of affine maps.MLIR Team2019-06-011-1/+1
| | | | | | -- PiperOrigin-RevId: 250572818
* Remove unused function and avoid unused variable warning. NFC.Jacques Pienaar2019-05-201-0/+1
| | | | | | -- PiperOrigin-RevId: 247991231
* Replace Operation::isa with llvm::isa.River Riddle2019-05-201-1/+1
| | | | | | -- PiperOrigin-RevId: 247789235
* Replace Operation::cast with llvm::cast.River Riddle2019-05-201-2/+2
| | | | | | -- PiperOrigin-RevId: 247785983
* Add support for using llvm::dyn_cast/cast/isa for operation casts and ↵River Riddle2019-05-201-4/+4
| | | | | | | | replace usages of Operation::dyn_cast with llvm::dyn_cast. -- PiperOrigin-RevId: 247780086
* Automated rollback of changelist 247778391.MLIR Team2019-05-201-4/+4
| | | | PiperOrigin-RevId: 247778691
* Add support for using llvm::dyn_cast/cast/isa for operation casts and ↵River Riddle2019-05-201-4/+4
| | | | | | | | replace usages of Operation::dyn_cast with llvm::dyn_cast. -- PiperOrigin-RevId: 247778391
* NFC: Replace usages of iterator_range<operand_iterator> with operand_range.River Riddle2019-04-051-1/+1
| | | | | | -- PiperOrigin-RevId: 242031201
* Cleanup SuperVectorization dialect printing and parsing.Nicolas Vasilache2019-03-291-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | On the read side, ``` %3 = vector_transfer_read %arg0, %i2, %i1, %i0 {permutation_map: (d0, d1, d2)->(d2, d0)} : (memref<?x?x?xf32>, index, index, index) -> vector<32x256xf32> ``` becomes: ``` %3 = vector_transfer_read %arg0[%i2, %i1, %i0] {permutation_map: (d0, d1, d2)->(d2, d0)} : memref<?x?x?xf32>, vector<32x256xf32> ``` On the write side, ``` vector_transfer_write %0, %arg0, %c3, %c3 {permutation_map: (d0, d1)->(d0)} : vector<128xf32>, memref<?x?xf32>, index, index ``` becomes ``` vector_transfer_write %0, %arg0[%c3, %c3] {permutation_map: (d0, d1)->(d0)} : vector<128xf32>, memref<?x?xf32> ``` Documentation will be cleaned up in a followup commit that also extracts a proper .md from the top of the file comments. PiperOrigin-RevId: 241021879
* Refactor vectorization patternsNicolas Vasilache2019-03-291-11/+17
| | | | | | | | | | This CL removes the reliance of the vectorize pass on the specification of a `fastestVaryingDim` parameter. This parameter is a restriction meant to more easily target a particular loop/memref combination for vectorization and is mainly used for testing. This also had the side-effect of restricting vectorization patterns to only the ones in which all memrefs were contiguous along the same loop dimension. This simple restriction prevented matmul to vectorize in 2-D. this CL removes the restriction and adds the matmul test which vectorizes in 2-D along the parallel loops. Support for reduction loops is left for future work. PiperOrigin-RevId: 240993827
* Replace usages of Instruction with Operation in the /Analysis directory.River Riddle2019-03-291-27/+26
| | | | PiperOrigin-RevId: 240569775
* Rename the Instruction class to Operation. This just renames the class, ↵River Riddle2019-03-291-1/+1
| | | | | | | | usages of Instruction will still refer to a typedef in the interim. This is step 1/N to renaming Instruction to Operation. PiperOrigin-RevId: 240431520
* Replace usages of "Op::operator->" with ".".River Riddle2019-03-291-4/+4
| | | | | | This is step 2/N of removing the temporary operator-> method as part of the de-const transition. PiperOrigin-RevId: 240200792
* Replace usages of "operator->" with "." for the AffineOps.River Riddle2019-03-291-1/+1
| | | | | Note: The "operator->" method is a temporary helper for the de-const transition and is gradually being phased out. PiperOrigin-RevId: 240179439
* Remove const from Value, Instruction, Argument, and the various methods on theChris Lattner2019-03-291-1/+1
| | | | | | *Op classes. This is a net reduction by almost 400LOC. PiperOrigin-RevId: 239972443
* NFC. Move all of the remaining operations left in BuiltinOps to StandardOps. ↵River Riddle2019-03-291-1/+0
| | | | | | The only thing left in BuiltinOps are the core MLIR types. The standard types can't be moved because they are referenced within the IR directory, e.g. in things like Builder. PiperOrigin-RevId: 236403665
* Use consistent names for dialect op source filesLei Zhang2019-03-291-1/+1
| | | | | | | | | | | | | | | This CL changes dialect op source files (.h, .cpp, .td) to follow the following convention: <full-dialect-name>/<dialect-namespace>Ops.{h|cpp|td} Builtin and standard dialects are specially treated, though. Both of them do not have dialect namespace; the former is still named as BuiltinOps.* and the latter is named as Ops.*. Purely mechanical. NFC. PiperOrigin-RevId: 236371358
* Begin the process of fully removing OperationInst. This patch cleans up ↵River Riddle2019-03-291-12/+10
| | | | | | references to OperationInst in the /include, /AffineOps, and lib/Analysis. PiperOrigin-RevId: 232199262
* Fold the functionality of OperationInst into Instruction. OperationInst ↵River Riddle2019-03-291-1/+1
| | | | | | still exists as a forward declaration and will be removed incrementally in a set of followup cleanup patches. PiperOrigin-RevId: 232198540
* Define the AffineForOp and replace ForInst with it. This patch is largely ↵River Riddle2019-03-291-17/+20
| | | | | | mechanical, i.e. changing usages of ForInst to OpPointer<AffineForOp>. An important difference is that upon construction an AffineForOp no longer automatically creates the body and induction variable. To generate the body/iv, 'createBody' can be called on an AffineForOp with no body. PiperOrigin-RevId: 232060516
* Change the ForInst induction variable to be a block argument of the body ↵River Riddle2019-03-291-1/+2
| | | | | | instead of the ForInst itself. This is a necessary step in converting ForInst into an operation. PiperOrigin-RevId: 231064139
* Migrate VectorOrTensorType/MemRefType shape api to use int64_t instead of int.River Riddle2019-03-291-4/+4
| | | | PiperOrigin-RevId: 230605756
* Move makeNormalizedAffineApplyNicolas Vasilache2019-03-291-207/+0
| | | | | | | | This CL is the 3rd on the path to simplifying AffineMap composition. This CL just moves `makeNormalizedAffineApply` from VectorAnalysis to AffineAnalysis where it more naturally belongs. PiperOrigin-RevId: 228277182
* Introduce AffineExpr::compose(AffineMap)Nicolas Vasilache2019-03-291-1/+2
| | | | | | | | | | | This CL is the 1st on the path to simplifying AffineMap composition. This CL uses the now accepted AffineExpr.replaceDimsAndSymbols to implement `AffineExpr::compose(AffineMap)`. Arguably, `simplifyAffineExpr` should be part of IR and not Analysis but this CL does not yet pull the trigger on that. PiperOrigin-RevId: 228265845
* Iterate on vector rather than DenseMap during AffineMap normalizationNicolas Vasilache2019-03-291-8/+10
| | | | | | | This CL removes a flakyness associated to a spurious iteration on DenseMap iterators when normalizing AffineMap. PiperOrigin-RevId: 228160074
* Fix opt build failureNicolas Vasilache2019-03-291-0/+1
| | | | PiperOrigin-RevId: 227938032
* [MLIR] Introduce normalized single-result unbounded AffineApplyOpNicolas Vasilache2019-03-291-0/+203
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Supervectorization does not plan on handling multi-result AffineMaps and non-canonical chains of > 1 AffineApplyOp. This CL introduces a simpler abstraction and composition of single-result unbounded AffineApplyOp by using the existing unbound AffineMap composition. This CL adds a simple API call and relevant tests: ```c++ OpPointer<AffineApplyOp> makeNormalizedAffineApply( FuncBuilder *b, Location loc, AffineMap map, ArrayRef<Value*> operands); ``` which creates a single-result unbounded AffineApplyOp. The operands of AffineApplyOp are not themselves results of AffineApplyOp by consrtuction. This represent the simplest possible interface to complement the composition of (mathematical) AffineMap, for the cases when we are interested in applying it to Value*. In this CL the composed AffineMap is not compressed (i.e. there exist operands that are not part of the result). A followup commit will compress to normal form. The single-result unbounded AffineApplyOp abstraction will be used in a followup CL to support the MaterializeVectors pass. PiperOrigin-RevId: 227879021
* [MLIR] Drop strict super-vector requirement in MaterializeVectorNicolas Vasilache2019-03-291-13/+5
| | | | | | | | | | | | | The strict requirement (i.e. at least 2 HW vectors in a super-vector) was a premature optimization to avoid interfering with other vector code potentially introduced via other means. This CL avoids this premature optimization and the spurious errors it causes when super-vector size == HW vector size (which is a possible corner case). This may be revisited in the future. PiperOrigin-RevId: 227763966
* Standardize naming of statements -> instructions, revisting the code base to beChris Lattner2019-03-291-27/+28
| | | | | | | | | consistent and moving the using declarations over. Hopefully this is the last truly massive patch in this refactoring. This is step 21/n towards merging instructions and statements, NFC. PiperOrigin-RevId: 227178245
* Merge Operation into OperationInst and standardize nomenclature aroundChris Lattner2019-03-291-3/+3
| | | | | | | | OperationInst. This is a big mechanical patch. This is step 16/n towards merging instructions and statements, NFC. PiperOrigin-RevId: 227093712
* Merge SSAValue, CFGValue, and MLValue together into a single Value class, whichChris Lattner2019-03-291-1/+1
| | | | | | | | | is the new base of the SSA value hierarchy. This CL also standardizes all the nomenclature and comments to use 'Value' where appropriate. This also eliminates a large number of cast<MLValue>(x)'s, which is very soothing. This is step 11/n towards merging instructions and statements, NFC. PiperOrigin-RevId: 227064624
* Extract vector_transfer_* Ops into a SuperVectorDialect.Alex Zinenko2019-03-291-0/+1
| | | | | | | | | | | From the beginning, vector_transfer_read and vector_transfer_write opreations were intended as a mid-level vectorization abstraction. In particular, they are lowered to the StandardOps dialect before further processing. As such, it does not make sense to keep them at the same level as StandardOps. Introduce the new SuperVectorOps dialect and move vector_transfer_* operations there. This will be used as a testbed for the generic lowering/legalization pass. PiperOrigin-RevId: 225554492
* [MLIR] Drop assert for NYI in VectorAnalysisNicolas Vasilache2019-03-291-6/+7
| | | | | | | This CLs adds proper error emission, removes NYI assertions and documents assumptions that are required in the relevant functions. PiperOrigin-RevId: 224377143
* [MLIR] Add support for permutation_mapNicolas Vasilache2019-03-291-11/+98
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This CL hooks up and uses permutation_map in vector_transfer ops. In particular, when going into the nuts and bolts of the implementation, it became clear that cases arose that required supporting broadcast semantics. Broadcast semantics are thus added to the general permutation_map. The verify methods and tests are updated accordingly. Examples of interest include. Example 1: The following MLIR snippet: ```mlir for %i3 = 0 to %M { for %i4 = 0 to %N { for %i5 = 0 to %P { %a5 = load %A[%i4, %i5, %i3] : memref<?x?x?xf32> }}} ``` may vectorize with {permutation_map: (d0, d1, d2) -> (d2, d1)} into: ```mlir for %i3 = 0 to %0 step 32 { for %i4 = 0 to %1 { for %i5 = 0 to %2 step 256 { %4 = vector_transfer_read %arg0, %i4, %i5, %i3 {permutation_map: (d0, d1, d2) -> (d2, d1)} : (memref<?x?x?xf32>, index, index) -> vector<32x256xf32> }}} ```` Meaning that vector_transfer_read will be responsible for reading the 2-D slice: `%arg0[%i4, %i5:%15+256, %i3:%i3+32]` into vector<32x256xf32>. This will require a transposition when vector_transfer_read is further lowered. Example 2: The following MLIR snippet: ```mlir %cst0 = constant 0 : index for %i0 = 0 to %M { %a0 = load %A[%cst0, %cst0] : memref<?x?xf32> } ``` may vectorize with {permutation_map: (d0) -> (0)} into: ```mlir for %i0 = 0 to %0 step 128 { %3 = vector_transfer_read %arg0, %c0_0, %c0_0 {permutation_map: (d0, d1) -> (0)} : (memref<?x?xf32>, index, index) -> vector<128xf32> } ```` Meaning that vector_transfer_read will be responsible of reading the 0-D slice `%arg0[%c0, %c0]` into vector<128xf32>. This will require a 1-D vector broadcast when vector_transfer_read is further lowered. Additionally, some minor cleanups and refactorings are performed. One notable thing missing here is the composition with a projection map during materialization. This is because I could not find an AffineMap composition that operates on AffineMap directly: everything related to composition seems to require going through SSAValue and only operates on AffinMap at a distance via AffineValueMap. I have raised this concern a bunch of times already, the followup CL will actually do something about it. In the meantime, the projection is hacked at a minimum to pass verification and materialiation tests are temporarily incorrect. PiperOrigin-RevId: 224376828
* [MLIR] Add VectorTransferOpsNicolas Vasilache2019-03-291-16/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This CL implements and uses VectorTransferOps in lieu of the former custom call op. Tests are updated accordingly. VectorTransferOps come in 2 flavors: VectorTransferReadOp and VectorTransferWriteOp. VectorTransferOps can be thought of as a backend-independent pseudo op/library call that needs to be legalized to MLIR (whiteboxed) before it can be lowered to backend-dependent IR. Note that the current implementation does not yet support a real permutation map. Proper support will come in a followup CL. VectorTransferReadOp ==================== VectorTransferReadOp performs a blocking read from a scalar memref location into a super-vector of the same elemental type. This operation is called 'read' by opposition to 'load' because the super-vector granularity is generally not representable with a single hardware register. As a consequence, memory transfers will generally be required when lowering VectorTransferReadOp. A VectorTransferReadOp is thus a mid-level abstraction that supports super-vectorization with non-effecting padding for full-tile only code. A vector transfer read has semantics similar to a vector load, with additional support for: 1. an optional value of the elemental type of the MemRef. This value supports non-effecting padding and is inserted in places where the vector read exceeds the MemRef bounds. If the value is not specified, the access is statically guaranteed to be within bounds; 2. an attribute of type AffineMap to specify a slice of the original MemRef access and its transposition into the super-vector shape. The permutation_map is an unbounded AffineMap that must represent a permutation from the MemRef dim space projected onto the vector dim space. Example: ```mlir %A = alloc(%size1, %size2, %size3, %size4) : memref<?x?x?x?xf32> ... %val = `ssa-value` : f32 // let %i, %j, %k, %l be ssa-values of type index %v0 = vector_transfer_read %src, %i, %j, %k, %l {permutation_map: (d0, d1, d2, d3) -> (d3, d1, d2)} : (memref<?x?x?x?xf32>, index, index, index, index) -> vector<16x32x64xf32> %v1 = vector_transfer_read %src, %i, %j, %k, %l, %val {permutation_map: (d0, d1, d2, d3) -> (d3, d1, d2)} : (memref<?x?x?x?xf32>, index, index, index, index, f32) -> vector<16x32x64xf32> ``` VectorTransferWriteOp ===================== VectorTransferWriteOp performs a blocking write from a super-vector to a scalar memref of the same elemental type. This operation is called 'write' by opposition to 'store' because the super-vector granularity is generally not representable with a single hardware register. As a consequence, memory transfers will generally be required when lowering VectorTransferWriteOp. A VectorTransferWriteOp is thus a mid-level abstraction that supports super-vectorization with non-effecting padding for full-tile only code. A vector transfer write has semantics similar to a vector store, with additional support for handling out-of-bounds situations. Example: ```mlir %A = alloc(%size1, %size2, %size3, %size4) : memref<?x?x?x?xf32>. %val = `ssa-value` : vector<16x32x64xf32> // let %i, %j, %k, %l be ssa-values of type index vector_transfer_write %val, %src, %i, %j, %k, %l {permutation_map: (d0, d1, d2, d3) -> (d3, d1, d2)} : (vector<16x32x64xf32>, memref<?x?x?x?xf32>, index, index, index, index) ``` PiperOrigin-RevId: 223873234
* [MLIR][Slicing] Add utils for computing slices.Nicolas Vasilache2019-03-291-27/+15
| | | | | | | | | | | | | | | | | | | | | | This CL adds tooling for computing slices as an independent CL. The first consumer of this analysis will be super-vector materialization in a followup CL. In particular, this adds: 1. a getForwardStaticSlice function with documentation, example and a standalone unit test; 2. a getBackwardStaticSlice function with documentation, example and a standalone unit test; 3. a getStaticSlice function with documentation, example and a standalone unit test; 4. a topologicalSort function that is exercised through the getStaticSlice unit test. The getXXXStaticSlice functions take an additional root (resp. terminators) parameter which acts as a boundary that the transitive propagation algorithm is not allowed to cross. PiperOrigin-RevId: 222446208
OpenPOWER on IntegriCloud