summaryrefslogtreecommitdiffstats
path: root/mlir/lib
Commit message (Collapse)AuthorAgeFilesLines
...
* Automated rollback of commit 98fbf41044d3364dbaf18db81b9e8d9520d14761Alexander Belyaev2019-12-112-383/+0
| | | | PiperOrigin-RevId: 284979684
* Add a function to get lowering patterns from GPU to NVVM.Stephan Herhut2019-12-111-13/+18
| | | | | | This enables combining the patterns with other patterns into larger lowerings. PiperOrigin-RevId: 284979271
* [Linalg] Add tiling for IndexedGenericOp with a region.Alexander Belyaev2019-12-111-9/+115
| | | | PiperOrigin-RevId: 284949355
* Add initial liveness analysis including test cases.Marcel Koester2019-12-112-0/+383
| | | | | | Closes tensorflow/mlir#255 PiperOrigin-RevId: 284935454
* [VectorOps] Add lowering of vector.insert to LLVM IRAart Bik2019-12-101-24/+100
| | | | | | | | | | | | | | | | | | | | | | | | | For example, an insert %0 = vector.insert %arg0, %arg1[3 : i32] : f32 into vector<4xf32> becomes %0 = llvm.mlir.constant(3 : i32) : !llvm.i32 %1 = llvm.insertelement %arg0, %arg1[%0 : !llvm.i32] : !llvm<"<4 x float>"> A more elaborate example, inserting an element in a higher dimension vector %0 = vector.insert %arg0, %arg1[3 : i32, 7 : i32, 15 : i32] : f32 into vector<4x8x16xf32> becomes %0 = llvm.extractvalue %arg1[3 : i32, 7 : i32] : !llvm<"[4 x [8 x <16 x float>]]"> %1 = llvm.mlir.constant(15 : i32) : !llvm.i32 %2 = llvm.insertelement %arg0, %0[%1 : !llvm.i32] : !llvm<"<16 x float>"> %3 = llvm.insertvalue %2, %arg1[3 : i32, 7 : i32] : !llvm<"[4 x [8 x <16 x float>]]"> PiperOrigin-RevId: 284882443
* Add VectorOp transform pattern which splits vector TransferReadOps to target ↵Andy Davis2019-12-102-6/+78
| | | | | | vector unroll size. PiperOrigin-RevId: 284880592
* More affine expr simplifications for floordiv and modUday Bondhugula2019-12-101-8/+35
| | | | | | | | | | | | | | | | | | | | | | Add one more simplification for floordiv and mod affine expressions. Examples: (2*d0 + 1) floordiv 2 is simplified to d0 (8*d0 + 4*d1 + d2) floordiv 4 simplified to 4*d0 + d1 + d2 floordiv 4. etc. Similarly, (4*d1 + 1) mod 2 is simplified to 1, (2*d0 + 8*d1) mod 8 simplified to 2*d0 mod 8. Change getLargestKnownDivisor to return int64_t to be consistent and to avoid casting at call sites (since the return value is used in expressions of int64_t/index type). Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Closes tensorflow/mlir#202 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/202 from bondhugula:affine b13fcb2f1c00a39ca5434613a02408e085a80e77 PiperOrigin-RevId: 284866710
* Move gpu.launch_func to ODS. NFCAlex Zinenko2019-12-101-14/+14
| | | | | | | | | Move the definition of gpu.launch_func operation from hand-rolled C++ implementation to the ODS framework. Also move the documentation. This only performs the move and remains a non-functional change, a follow-up will clean up the custom functions that can be auto-generated using ODS. PiperOrigin-RevId: 284842252
* Refactor the various operand/result/type iterators to use ↵River Riddle2019-12-104-57/+67
| | | | | | | | | | | | | | indexed_accessor_range. This has several benefits: * The implementation is much cleaner and more efficient. * The ranges now have support for many useful operations: operator[], slice, drop_front, size, etc. * Value ranges can now directly query a range for their types via 'getTypes()': e.g: void foo(Operation::operand_range operands) { auto operandTypes = operands.getTypes(); } PiperOrigin-RevId: 284834912
* [Linalg] Add a Linalg iterator permutation transformationJose Ignacio Gomez2019-12-102-9/+38
| | | | | | | | | | | | | | | This patch closes issue tensorflow/mlir#272 We add a standalone iterator permutation transformation to Linalg. This transformation composes a permutation map with the maps in the "indexing_maps" attribute. It also permutes "iterator_types" accordingly. Change-Id: I7c1e693b8203aeecc595a7c012e738ca1100c857 Closes tensorflow/mlir#307 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/307 from tetuante:issue272 f7908d58792f4111119721885e247045104f1131 PiperOrigin-RevId: 284824102
* Uniformize Vector transforms as patterns on the model of Linalg - NFCNicolas Vasilache2019-12-103-24/+5
| | | | | | This reorganizes the vector transformations to be more easily testable as patterns and more easily composable into fused passes in the future. PiperOrigin-RevId: 284817474
* More convenience build methods for SPIR-V ops.Mahesh Ravishankar2019-12-102-61/+85
| | | | | | | | | | | | Add some convenience build methods to SPIR-V ops and update the lowering to use these methods where possible. For SPIRV::CompositeExtractOp move the method to deduce type of element based on base and indices into a convenience function. Some additional functionality needed to handle differences between parsing and verification methods. PiperOrigin-RevId: 284794404
* Drop Markdown style annotationsAlex Zinenko2019-12-101-2/+2
| | | | | | | These come from a non-standard extenion that is not available on Github, so it only clutters the documentation source with {.mlir} or {.ebnf} tags. PiperOrigin-RevId: 284733003
* [VectorOps] Add a ShuffleOp to the VectorOps dialectAart Bik2019-12-091-0/+86
| | | | | | | | | | For example %0 = vector.shuffle %x, %y [3 : i32, 2 : i32, 1 : i32, 0 : i32] : vector<2xf32>, vector<2xf32> yields a vector<4xf32> result with a permutation of the elements of %x and %y PiperOrigin-RevId: 284657191
* [VectorOps] Fix off-by-one error in insert/extract validationAart Bik2019-12-091-2/+2
| | | | PiperOrigin-RevId: 284652653
* Refactor the Block support classes.River Riddle2019-12-091-0/+10
| | | | | | Each of the support classes for Block are now moved into a new header BlockSupport.h. The successor iterator class is also reimplemented as an indexed_accessor_range. This makes the class more efficient, and expands on its available functionality. PiperOrigin-RevId: 284646792
* Add new indexed_accessor_range_base and indexed_accessor_range classes that ↵River Riddle2019-12-093-49/+36
| | | | | | | | simplify defining index-able ranges. Many ranges want similar functionality from a range type(e.g. slice/drop_front/operator[]/etc.), so these classes provide a generic implementation that may be used by many different types of ranges. This removes some code duplication, and also empowers many of the existing range types in MLIR(e.g. result type ranges, operand ranges, ElementsAttr ranges, etc.). This change only updates RegionRange and ValueRange, more ranges will be updated in followup commits. PiperOrigin-RevId: 284615679
* [spirv] Add CompositeConstruct operation.Denis Khalikov2019-12-091-0/+67
| | | | | | | Closes tensorflow/mlir#308 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/308 from denis0x0D:sandbox/composite_construct 9ef7180f77f9374bcd05afc4f9e6c1d2d72d02b7 PiperOrigin-RevId: 284613617
* [spirv] Add spv.IAdd, spv.ISub, and spv.IMul foldersLei Zhang2019-12-091-5/+36
| | | | | | | The patterns to be folded away can be commonly generated during lowering to SPIR-V. PiperOrigin-RevId: 284604855
* Factor out commonly reusable names across structured ops dialectsNicolas Vasilache2019-12-091-3/+5
| | | | | | This CL starts extracting commonalities between dialects that use the structured ops abstractions. Also fixes an OSS build issue where StringRef were incorrectly used with constexpr. PiperOrigin-RevId: 284591114
* Add lowering for module with gpu.kernel_module attribute.Mahesh Ravishankar2019-12-094-36/+136
| | | | | | | | | | | The existing GPU to SPIR-V lowering created a spv.module for every function with gpu.kernel attribute. A better approach is to lower the module that the function lives in (which has the attribute gpu.kernel_module) to a spv.module operation. This better captures the host-device separation modeled by GPU dialect and simplifies the lowering as well. PiperOrigin-RevId: 284574688
* Unify vector op unrolling transformation.Andy Davis2019-12-091-394/+175
| | | | | | | Unifies vector op unrolling transformation, by using the same unrolling implementation for contraction and elementwise operations. Removes fakefork/join operations which are non longer needed now that we have the InsertStridedSlice operation. PiperOrigin-RevId: 284570784
* Minor spelling tweaksKazuaki Ishizaki2019-12-098-10/+10
| | | | | | Closes tensorflow/mlir#304 PiperOrigin-RevId: 284568358
* [StructuredOps][Linalg] Add a primitive pattern to rewrite the ↵Nicolas Vasilache2019-12-093-2/+102
| | | | | | | | | | | linalg.generic form of matmul to vector form. This CL uses the newly expanded matcher support to easily detect when a linalg.generic has a multiply-accumulate body. A linalg.generic with such a body is rewritten as a vector contraction. This CL additionally limits the rewrite to the case of matrix multiplication on contiguous and statically shaped memrefs for now. Before expanding further, we should harden the infrastructure for expressing custom ops with the structured ops abstraction. PiperOrigin-RevId: 284566659
* Add RegionRange for when need to abstract over different region iterationJacques Pienaar2019-12-092-2/+19
| | | | | | | | | | | | Follows ValueRange in representing a generic abstraction over the different ways to represent a range of Regions. This wrapper is not as ValueRange and only considers the current cases of interest: MutableArrayRef<Region> and ArrayRef<std::unique_ptr<Region>> as occurs during op construction vs op region querying. Note: ArrayRef<std::unique_ptr<Region>> allows for unset regions, so this range returns a pointer to a Region instead of a Region. PiperOrigin-RevId: 284563229
* Replace spurious SmallVector constructions with ValueRangeUday Bondhugula2019-12-094-11/+9
| | | | | | | | | Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Closes tensorflow/mlir#305 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/305 from bondhugula:value_range 21d1fae73f549e3c8e72b60876eff1b864cea39c PiperOrigin-RevId: 284541027
* NFC: Expose constFoldBinaryOp via a headerLei Zhang2019-12-081-48/+1
| | | | | | | This allows other dialects to reuse the logic to support constant folding binary operations and reduces code duplication. PiperOrigin-RevId: 284428721
* Update the builder API to take ValueRange instead of ArrayRef<Value *>River Riddle2019-12-0714-101/+118
| | | | | | This allows for users to provide operand_range and result_range in builder.create<> calls, instead of requiring an explicit copy into a separate data structure like SmallVector/std::vector. PiperOrigin-RevId: 284360710
* Add a new ValueRange class.River Riddle2019-12-068-47/+65
| | | | | | | | This class represents a generic abstraction over the different ways to represent a range of Values: ArrayRef<Value *>, operand_range, result_range. This class will allow for removing the many instances of explicit SmallVector<Value *, N> construction. It has the same memory cost as ArrayRef, and only suffers cost from indexing(if+elsing the different underlying representations). This change only updates a few of the existing usages, with more to be changed in followups; e.g. 'build' API. PiperOrigin-RevId: 284307996
* Add a flag to the IRPrinter instrumentation to only print after a pass if ↵River Riddle2019-12-062-9/+99
| | | | | | | | there is a change to the IR. This adds an additional filtering mode for printing after a pass that checks to see if the pass actually changed the IR before printing it. This "change" detection is implemented using a SHA1 hash of the current operation and its children. PiperOrigin-RevId: 284291089
* NFC - update doc, comments, vim syntax fileUday Bondhugula2019-12-062-2/+9
| | | | | | | | | | | - for the symbol rules, the code was updated but the doc wasn't. Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Closes tensorflow/mlir#284 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/284 from bondhugula:doc 9aad8b8a715559f7ce61265f3da3f8a3c11b45ea PiperOrigin-RevId: 284283712
* NFC: Separate implementation and definition in ConvertStandardToSPIRV.cppMahesh Ravishankar2019-12-061-140/+188
| | | | PiperOrigin-RevId: 284274326
* Replace custom getBody method with an ODS-generated in gpu::LaunchOpAlex Zinenko2019-12-063-28/+26
| | | | PiperOrigin-RevId: 284262981
* During serialization do a walk of ops in module to find spv.module.Mahesh Ravishankar2019-12-061-4/+5
| | | | | | | | During lowering, spv.module might be within other modules (for example gpu kernel module). Walk the module op to find spirv module to serialize. PiperOrigin-RevId: 284262550
* Move GPU::LaunchOp to ODS. NFC.Alex Zinenko2019-12-061-29/+35
| | | | | | | | Move the definition of the GPU launch opreation from hand-rolled C++ code to ODS framework. This only does the moves, a follow-up is necessary to clean up users of custom functions that could be auto-generated by ODS. PiperOrigin-RevId: 284261856
* [VecOps] Rename vector.[insert|extract]element to just vector.[insert|extract]Aart Bik2019-12-062-28/+22
| | | | | | | Since these operations lower to [insert|extract][element|value] at LLVM dialect level, neither element nor value would correctly reflect the meaning. PiperOrigin-RevId: 284240727
* LLVM::GlobalOp: take address space as builder argumentAlex Zinenko2019-12-062-8/+6
| | | | | | | | | Accept the address space of the global as a builder argument when constructing an LLVM::GlobalOp instance. This decreases the reliance of LLVM::GlobalOp users on the internal name of the attribute used for this purpose. Update several uses of the address space in GPU to NVVM conversion. PiperOrigin-RevId: 284233254
* Move GPU::FuncOp definition to ODS - NFCAlex Zinenko2019-12-061-29/+30
| | | | | | | | Move the definition of the GPU function opreation from hand-rolled C++ code to ODS framework. This only does the moves, a follow-up is necessary to clean up users of custom functions that could be auto-generated by ODS. PiperOrigin-RevId: 284233245
* [VectorOps] Add lowering of vector.broadcast to LLVM IRAart Bik2019-12-062-7/+192
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For example, a scalar broadcast %0 = vector.broadcast %x : f32 to vector<2xf32> return %0 : vector<2xf32> which expands scalar x into vector [x,x] by lowering to the following LLVM IR dialect to implement the duplication over the leading dimension. %0 = llvm.mlir.undef : !llvm<"<2 x float>"> %1 = llvm.mlir.constant(0 : index) : !llvm.i64 %2 = llvm.insertelement %x, %0[%1 : !llvm.i64] : !llvm<"<2 x float>"> %3 = llvm.shufflevector %2, %0 [0 : i32, 0 : i32] : !llvm<"<2 x float>">, !llvm<"<2 x float>"> return %3 : vector<2xf32> In the trailing dimensions, the operand is simply "passed through", unless a more elaborate "stretch" is required. For example %0 = vector.broadcast %arg0 : vector<1xf32> to vector<4xf32> return %0 : vector<4xf32> becomes %0 = llvm.mlir.undef : !llvm<"<4 x float>"> %1 = llvm.mlir.constant(0 : index) : !llvm.i64 %2 = llvm.extractelement %arg0[%1 : !llvm.i64] : !llvm<"<1 x float>"> %3 = llvm.mlir.constant(0 : index) : !llvm.i64 %4 = llvm.insertelement %2, %0[%3 : !llvm.i64] : !llvm<"<4 x float>"> %5 = llvm.shufflevector %4, %0 [0 : i32, 0 : i32, 0 : i32, 0 : i32] : !llvm<"<4 x float>">, !llvm<"<4 x float>"> llvm.return %5 : !llvm<"<4 x float>"> PiperOrigin-RevId: 284219926
* Add conversions of GPU func with memory attributions to LLVM/NVVMAlex Zinenko2019-12-062-3/+204
| | | | | | | | | | | | | | | | | | | GPU functions use memory attributions, a combination of Op attributes and region arguments, to specify function-wide buffers placed in workgroup or private memory spaces. Introduce a lowering pattern for GPU functions to be converted to LLVM functions taking into account memory attributions. Workgroup attributions get transformed into module-level globals with unique names derived from function names. Private attributions get converted into llvm.allocas inside the function body. In both cases, we inject at the beginning of the function the IR that obtains the raw pointer to the data and populates a MemRef descriptor based on the MemRef type of buffer, making attributions compose with the rest of the MemRef lowering and transparent for use with std.load and std.store. While using raw pointers instead of descriptors might have been more efficient, it is better implemented as a canonicalization or a separate transformation so that non-attribution memrefs could also benefit from it. PiperOrigin-RevId: 284208396
* fix examples in commentsAlexandre E. Eichenberger2019-12-061-20/+18
| | | | | | | Closes tensorflow/mlir#301 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/301 from AlexandreEichenberger:vect-doc-update 7e5418a9101a4bdad2357882fe660b02bba8bd01 PiperOrigin-RevId: 284202462
* Unroll vector masks along with their associated vector arguments.Andy Davis2019-12-062-59/+32
| | | | | | | | | Updates vector ContractionOp to use proper vector masks (produced by CreateMaskOp/ConstantMaskOp). Leverages the following canonicalizations in unrolling unit test: CreateMaskOp -> ConstantMaskOp, StridedSliceOp(ConstantMaskOp) -> ConstantMaskOp Removes IndexTupleOp (no longer needed now that we have vector mask ops). Updates all unit tests. PiperOrigin-RevId: 284182168
* [spirv] Reorder `erase` and `emplace` to avoid "invalid iterator access".Denis Khalikov2019-12-061-1/+3
| | | | | | | | | | The iterator should be erased before adding a new entry into blockMergeInfo to avoid iterator invalidation. Closes tensorflow/mlir#299 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/299 from denis0x0D:sandbox/reoder_erase 983be565809aa0aadfc7e92962e4d4b282f63c66 PiperOrigin-RevId: 284173235
* DimOp folding for alloc/view dynamic dimensionsUday Bondhugula2019-12-061-2/+17
| | | | | | | | | Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Closes tensorflow/mlir#253 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/253 from bondhugula:dimop a4b464f24ae63fd259114558d87e11b8ee4dae86 PiperOrigin-RevId: 284169689
* minor spelling tweaksKazuaki Ishizaki2019-12-069-14/+13
| | | | | | | Closes tensorflow/mlir#290 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/290 from kiszk:spelling_tweaks_201912 9d9afd16a723dd65754a04698b3976f150a6054a PiperOrigin-RevId: 284169681
* LLVM::AddressOfOp: properly take into account the address spaceAlex Zinenko2019-12-061-1/+2
| | | | | | | | | | The AddressOf operation in the LLVM dialect return a pointer to a global variable. The latter may be in a non-default address space as indicated by the "addr_space" attribute. Check that the address space of the pointer returned by AddressOfOp matches that of the referenced GlobalOp. Update the AddressOfOp builder to respect this constraint. PiperOrigin-RevId: 284138860
* [Linalg] Add permutation information to tilingJose Ignacio Gomez2019-12-053-15/+56
| | | | | | | | | | | This patch closes issue tensorflow/mlir#271. It adds an optional permutation map to declarative tiling transformations. The map is expressed as a list of integers. Closes tensorflow/mlir#288 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/288 from tetuante:issue271 2df2938d6a1f01b3bc404ded08dea2dd1e10b588 PiperOrigin-RevId: 284064151
* Refactor the IRPrinting instrumentation to take a derivable config.River Riddle2019-12-052-48/+104
| | | | | | This allows for more interesting behavior from users, e.g. enabling the ability to dump the IR to a separate file for each pass invocation. PiperOrigin-RevId: 284059447
* Add UnrankedMemRef Typenmostafa2019-12-057-70/+312
| | | | | | | Closes tensorflow/mlir#261 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/261 from nmostafa:nmostafa/unranked 96b6e918f6ed64496f7573b2db33c0b02658ca45 PiperOrigin-RevId: 284037040
* [spirv] Add CompositeInsertOp operationDenis Khalikov2019-12-051-21/+86
| | | | | | | | | | A CompositeInsertOp operation make a copy of a composite object, while modifying one part of it. Closes tensorflow/mlir#292 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/292 from denis0x0D:sandbox/composite_insert 2200962b9057bda53cd2f2866b461e2797196380 PiperOrigin-RevId: 284036551
OpenPOWER on IntegriCloud