summaryrefslogtreecommitdiffstats
path: root/mlir/lib/Conversion
Commit message (Collapse)AuthorAgeFilesLines
...
* Switch from shfl.bfly to shfl.down.Christian Sigg2019-12-121-2/+2
| | | | | | | Both work for the current use case, but the latter allows implementing prefix sums and is a little easier to understand for partial warps. PiperOrigin-RevId: 285145287
* Add std.log* and llvm.intr.log* that correspond to the LLVMIR intrinsicsNicolas Vasilache2019-12-111-0/+12
| | | | PiperOrigin-RevId: 285073483
* Expose a convenience function to add interface attributes to a function.Mahesh Ravishankar2019-12-111-1/+1
| | | | PiperOrigin-RevId: 285036647
* [spirv] Add lowering for std.fdiv, std.frem, std.fsubDenis Khalikov2019-12-111-0/+3
| | | | | | | Closes tensorflow/mlir#313 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/313 from denis0x0D:sandbox/lowering_std_farith 41715070a74d13bfa9401957478978c1bb8006c0 PiperOrigin-RevId: 285023586
* NFC: Fix naming inconsistency: FuncOpLowering -> GPUFuncOpLowering.Christian Sigg2019-12-111-7/+3
| | | | | | Remove nested anonymous namespace. PiperOrigin-RevId: 284987357
* Add a function to get lowering patterns from GPU to NVVM.Stephan Herhut2019-12-111-13/+18
| | | | | | This enables combining the patterns with other patterns into larger lowerings. PiperOrigin-RevId: 284979271
* [VectorOps] Add lowering of vector.insert to LLVM IRAart Bik2019-12-101-24/+100
| | | | | | | | | | | | | | | | | | | | | | | | | For example, an insert %0 = vector.insert %arg0, %arg1[3 : i32] : f32 into vector<4xf32> becomes %0 = llvm.mlir.constant(3 : i32) : !llvm.i32 %1 = llvm.insertelement %arg0, %arg1[%0 : !llvm.i32] : !llvm<"<4 x float>"> A more elaborate example, inserting an element in a higher dimension vector %0 = vector.insert %arg0, %arg1[3 : i32, 7 : i32, 15 : i32] : f32 into vector<4x8x16xf32> becomes %0 = llvm.extractvalue %arg1[3 : i32, 7 : i32] : !llvm<"[4 x [8 x <16 x float>]]"> %1 = llvm.mlir.constant(15 : i32) : !llvm.i32 %2 = llvm.insertelement %arg0, %0[%1 : !llvm.i32] : !llvm<"<16 x float>"> %3 = llvm.insertvalue %2, %arg1[3 : i32, 7 : i32] : !llvm<"[4 x [8 x <16 x float>]]"> PiperOrigin-RevId: 284882443
* Drop Markdown style annotationsAlex Zinenko2019-12-101-2/+2
| | | | | | | These come from a non-standard extenion that is not available on Github, so it only clutters the documentation source with {.mlir} or {.ebnf} tags. PiperOrigin-RevId: 284733003
* Add lowering for module with gpu.kernel_module attribute.Mahesh Ravishankar2019-12-092-27/+94
| | | | | | | | | | | The existing GPU to SPIR-V lowering created a spv.module for every function with gpu.kernel attribute. A better approach is to lower the module that the function lives in (which has the attribute gpu.kernel_module) to a spv.module operation. This better captures the host-device separation modeled by GPU dialect and simplifies the lowering as well. PiperOrigin-RevId: 284574688
* Minor spelling tweaksKazuaki Ishizaki2019-12-091-3/+3
| | | | | | Closes tensorflow/mlir#304 PiperOrigin-RevId: 284568358
* Replace spurious SmallVector constructions with ValueRangeUday Bondhugula2019-12-091-5/+5
| | | | | | | | | Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Closes tensorflow/mlir#305 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/305 from bondhugula:value_range 21d1fae73f549e3c8e72b60876eff1b864cea39c PiperOrigin-RevId: 284541027
* Update the builder API to take ValueRange instead of ArrayRef<Value *>River Riddle2019-12-074-23/+17
| | | | | | This allows for users to provide operand_range and result_range in builder.create<> calls, instead of requiring an explicit copy into a separate data structure like SmallVector/std::vector. PiperOrigin-RevId: 284360710
* Add a new ValueRange class.River Riddle2019-12-061-2/+1
| | | | | | | | This class represents a generic abstraction over the different ways to represent a range of Values: ArrayRef<Value *>, operand_range, result_range. This class will allow for removing the many instances of explicit SmallVector<Value *, N> construction. It has the same memory cost as ArrayRef, and only suffers cost from indexing(if+elsing the different underlying representations). This change only updates a few of the existing usages, with more to be changed in followups; e.g. 'build' API. PiperOrigin-RevId: 284307996
* NFC: Separate implementation and definition in ConvertStandardToSPIRV.cppMahesh Ravishankar2019-12-061-140/+188
| | | | PiperOrigin-RevId: 284274326
* Replace custom getBody method with an ODS-generated in gpu::LaunchOpAlex Zinenko2019-12-061-7/+7
| | | | PiperOrigin-RevId: 284262981
* [VecOps] Rename vector.[insert|extract]element to just vector.[insert|extract]Aart Bik2019-12-061-3/+3
| | | | | | | Since these operations lower to [insert|extract][element|value] at LLVM dialect level, neither element nor value would correctly reflect the meaning. PiperOrigin-RevId: 284240727
* LLVM::GlobalOp: take address space as builder argumentAlex Zinenko2019-12-061-7/+2
| | | | | | | | | Accept the address space of the global as a builder argument when constructing an LLVM::GlobalOp instance. This decreases the reliance of LLVM::GlobalOp users on the internal name of the attribute used for this purpose. Update several uses of the address space in GPU to NVVM conversion. PiperOrigin-RevId: 284233254
* [VectorOps] Add lowering of vector.broadcast to LLVM IRAart Bik2019-12-061-1/+186
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For example, a scalar broadcast %0 = vector.broadcast %x : f32 to vector<2xf32> return %0 : vector<2xf32> which expands scalar x into vector [x,x] by lowering to the following LLVM IR dialect to implement the duplication over the leading dimension. %0 = llvm.mlir.undef : !llvm<"<2 x float>"> %1 = llvm.mlir.constant(0 : index) : !llvm.i64 %2 = llvm.insertelement %x, %0[%1 : !llvm.i64] : !llvm<"<2 x float>"> %3 = llvm.shufflevector %2, %0 [0 : i32, 0 : i32] : !llvm<"<2 x float>">, !llvm<"<2 x float>"> return %3 : vector<2xf32> In the trailing dimensions, the operand is simply "passed through", unless a more elaborate "stretch" is required. For example %0 = vector.broadcast %arg0 : vector<1xf32> to vector<4xf32> return %0 : vector<4xf32> becomes %0 = llvm.mlir.undef : !llvm<"<4 x float>"> %1 = llvm.mlir.constant(0 : index) : !llvm.i64 %2 = llvm.extractelement %arg0[%1 : !llvm.i64] : !llvm<"<1 x float>"> %3 = llvm.mlir.constant(0 : index) : !llvm.i64 %4 = llvm.insertelement %2, %0[%3 : !llvm.i64] : !llvm<"<4 x float>"> %5 = llvm.shufflevector %4, %0 [0 : i32, 0 : i32, 0 : i32, 0 : i32] : !llvm<"<4 x float>">, !llvm<"<4 x float>"> llvm.return %5 : !llvm<"<4 x float>"> PiperOrigin-RevId: 284219926
* Add conversions of GPU func with memory attributions to LLVM/NVVMAlex Zinenko2019-12-062-3/+204
| | | | | | | | | | | | | | | | | | | GPU functions use memory attributions, a combination of Op attributes and region arguments, to specify function-wide buffers placed in workgroup or private memory spaces. Introduce a lowering pattern for GPU functions to be converted to LLVM functions taking into account memory attributions. Workgroup attributions get transformed into module-level globals with unique names derived from function names. Private attributions get converted into llvm.allocas inside the function body. In both cases, we inject at the beginning of the function the IR that obtains the raw pointer to the data and populates a MemRef descriptor based on the MemRef type of buffer, making attributions compose with the rest of the MemRef lowering and transparent for use with std.load and std.store. While using raw pointers instead of descriptors might have been more efficient, it is better implemented as a canonicalization or a separate transformation so that non-attribution memrefs could also benefit from it. PiperOrigin-RevId: 284208396
* minor spelling tweaksKazuaki Ishizaki2019-12-061-1/+1
| | | | | | | Closes tensorflow/mlir#290 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/290 from kiszk:spelling_tweaks_201912 9d9afd16a723dd65754a04698b3976f150a6054a PiperOrigin-RevId: 284169681
* Add UnrankedMemRef Typenmostafa2019-12-051-31/+164
| | | | | | | Closes tensorflow/mlir#261 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/261 from nmostafa:nmostafa/unranked 96b6e918f6ed64496f7573b2db33c0b02658ca45 PiperOrigin-RevId: 284037040
* Allow specification of the workgroup size for GPUToSPIRV lowering.Mahesh Ravishankar2019-12-053-11/+48
| | | | | | | | | | SPIR-V/Vulkan spec requires the workgroups size to be specified with the spv.ExecutionMode operation. This was hard-wired to be set to a particular value. It is now changed to be configurable by clients of the pass or of the patterns that implement the lowering from GPU to SPIRV. PiperOrigin-RevId: 284017482
* Add a CL option to Standard to LLVM lowering to use alloca instead of ↵Nicolas Vasilache2019-12-041-35/+89
| | | | | | | | | malloc/free. In the future, a more configurable malloc and free interface should be used and exposed via extra parameters to the `createLowerToLLVMPass`. Until requirements are gathered, a simple CL flag allows generating code that runs successfully on hardware that cannot use the stdlib. PiperOrigin-RevId: 283833424
* Refactor dependencies to expose Vector transformations as patterns - NFCNicolas Vasilache2019-12-038-482/+37
| | | | | | | | | This CL refactors some of the MLIR vector dependencies to allow decoupling VectorOps, vector analysis, vector transformations and vector conversions from each other. This makes the system more modular and allows extracting VectorToVector into VectorTransforms that do not depend on vector conversions. This refactoring exhibited a bunch of cyclic library dependencies that have been cleaned up. PiperOrigin-RevId: 283660308
* Add a pass to legalize operations before lowering to SPIR-V.Mahesh Ravishankar2019-12-033-1/+194
| | | | | | | | | | | | Not all StandardOps can be lowered to SPIR-V. For example, subview op implementation requires use of pointer bitcasts which is not valid according to SPIR-V spec (or at least is ambiguous about it). Such ops need to be removed/transformed before lowering to SPIR-V. The SPIRVLegalizationPass is added a place where such legalizations can be added. Current implementation folds the subview ops with load/stores so that the lowering itself does not have to convert a subview op. PiperOrigin-RevId: 283642981
* Convert MemRefType to a linearized array in SPIR-V lowering.Mahesh Ravishankar2019-12-031-37/+57
| | | | | | | | | | | | | | | The SPIR-V lowering used nested !spv.arrays to represented multi-dimensional arrays, with the hope that in-conjunction with the layout annotations, the shape and layout of memref can be represented directly. It is unclear though how portable this representation will end up being. It will rely on driver compilers implementing complex index computations faithfully. A more portable approach is to use linearized arrays to represent memrefs and explicitly instantiate all the index computation in SPIR-V. This gives added benefit that we can further optimize the generated code in MLIR before generating the SPIR-V binary. PiperOrigin-RevId: 283571167
* Fix ViewOp to have at most one offset operandAlex Zinenko2019-12-031-6/+11
| | | | | | | | | | | | | | | | | | | | | As described in the documentation, ViewOp is expected to take an optional dynamic offset followed by a list of dynamic sizes. However, the ViewOp parser did not include a check for the offset being a single value and accepeted a list of values instead. Furthermore, several tests have been exercising the wrong syntax of a ViewOp, passing multiple values to the dyanmic stride list, which was not caught by the parser. The trailing values could have been erronously interpreted as dynamic sizes. This is likely due to resyntaxing of the ViewOp, with the previous syntax taking the list of sizes before the offset. Update the tests to use the syntax with the offset preceding the sizes. Worse, the conversion of ViewOp to the LLVM dialect assumed the wrong order of operands with offset in the trailing position, and erronously relied on the permissive parsing that interpreted trailing dynamic offset values as leading dynamic sizes. Fix the lowering to use the correct order of operands. PiperOrigin-RevId: 283532506
* Extend conversion of SubViewOp to llvm to also support cases where size and ↵Stephan Herhut2019-12-031-6/+24
| | | | | | | | | | | | stride are constant (i.e., there are no size and stride operands). We recently added canonicalization that rewrites constant size and stride operands to SubViewOp into static information in the type, so these patterns now occur during code generation. PiperOrigin-RevId: 283524688
* Add linkage support to LLVMFuncOpAlex Zinenko2019-12-032-3/+5
| | | | | | | | | A recent commit introduced the Linkage attribute to the LLVM dialect and used it in the Global Op. Also use it in LLVMFuncOp. As per LLVM Language Reference, if the linkage attribute is omitted, the function is assumed to have external linkage. PiperOrigin-RevId: 283493299
* NFC: Update std.subview op to use AttrSizedOperandSegmentsLei Zhang2019-12-021-2/+1
| | | | | | This turns a few manually written helper methods into auto-generated ones. PiperOrigin-RevId: 283339617
* Lower linalg.indexed_generic with libcall to LLVM.Alexander Belyaev2019-12-021-1/+57
| | | | PiperOrigin-RevId: 283328994
* Introduce Linkage attribute to the LLVM dialectAlex Zinenko2019-12-022-4/+5
| | | | | | | | | | | LLVM IR supports linkage on global objects such as global variables and functions. Introduce the Linkage attribute into the LLVM dialect, backed by an integer storage. Use this attribute on LLVM::GlobalOp and make it mandatory. Implement parsing/printing of the attribute and conversion to LLVM IR. See tensorflow/mlir#277. PiperOrigin-RevId: 283309328
* [spirv] NFC: Add getZero() and getOne() static method to ConstantOpLei Zhang2019-11-271-2/+1
| | | | | | | Getting constant zero or one is very common so it merits a special handy method on spirv::ConstantOp itself. PiperOrigin-RevId: 282832572
* Misc changes to lowering to SPIR-V.Mahesh Ravishankar2019-11-262-19/+76
| | | | | | | | | | | | | | | These changes to SPIR-V lowering while adding support for lowering SUbViewOp, but are not directly related. - Change the lowering of MemRefType to !spv.ptr<!spv.struct<!spv.array<...>[offset]>, ..> This is consistent with the Vulkan spec. - To enable testing a simple pattern of lowering functions is added to ConvertStandardToSPIRVPass. This is just used to convert the type of the arguments of the function. The added function lowering itself is not meant to be the way functions are eventually lowered into SPIR-V dialect. PiperOrigin-RevId: 282589644
* Use vector.InsertStridedSlice in Vector -> Vector unrollingNicolas Vasilache2019-11-251-8/+70
| | | | | | | | This CL uses the recently added op to finish the implementation of Vector -> Vector unrolling by replacing the "fake join op" by a series of InsertStridedSliceOp. Test is updated accordingly PiperOrigin-RevId: 282451126
* NFC: Actually expose the implementation of createGPUToSPIRVLoweringPass.Mahesh Ravishankar2019-11-251-2/+3
| | | | | | | | A mismatch in the function declaration and function definition, prevented the implementation of the createGPUToSPIRVLoweringPass from being exposed. PiperOrigin-RevId: 282419815
* Introduce attributes that specify the final ABI for a spirv::ModuleOp.Mahesh Ravishankar2019-11-254-36/+21
| | | | | | | | | | | | | | | | | | | | | | | | To simplify the lowering into SPIR-V, while still respecting the ABI requirements of SPIR-V/Vulkan, split the process into two 1) While lowering a function to SPIR-V (when the function is an entry point function), allow specifying attributes on arguments and function itself that describe the ABI of the function. 2) Add a pass that materializes the ABI described in the function. Two attributes are needed. 1) Attribute on arguments of the entry point function that describe the descriptor_set, binding, storage class, etc, of the spv.globalVariable this argument will be replaced by 2) Attribute on function that specifies workgroup size, etc. (for now only workgroup size). Add the pass -spirv-lower-abi-attrs to materialize the ABI described by the attributes. This change makes the SPIRVBasicTypeConverter class unnecessary and is removed, further simplifying the SPIR-V lowering path. PiperOrigin-RevId: 282387587
* NFC: Remove unnecessarily guarded tablegen includes.River Riddle2019-11-222-10/+0
| | | | | | Support for including a file multiple times was added in tablegen, removing the need for these extra guards. This is because we already insert c/c++ style header guards within each of the specific .td files. PiperOrigin-RevId: 282076728
* Unify vector op names with other dialects.Jean-Michel Gorius2019-11-223-37/+34
| | | | | | | | | | Change vector op names from VectorFooOp to Vector_FooOp and from vector::VectorFooOp to vector::FooOp. Closes tensorflow/mlir#257 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/257 from Kayjukh:master dfc3a0e04114885aaec8740d5951d6984d6e1577 PiperOrigin-RevId: 281967461
* Move Linalg Transforms that are actually Conversions - NFCNicolas Vasilache2019-11-213-0/+537
| | | | PiperOrigin-RevId: 281844602
* Changes to SubViewOp to make it more amenable to canonicalization.Mahesh Ravishankar2019-11-201-5/+20
| | | | | | | | | | | | | | The current SubViewOp specification allows for either all offsets, shape and stride to be dynamic or all of them to be static. There are opportunities for more fine-grained canonicalization based on which of these are static. For example, if the sizes are static, the result memref is of static shape. The specification of SubViewOp is modified to allow on or more of offsets, shapes and strides to be statically specified. The verification is updated to ensure that the result type of the subview op is consistent with which of these are static and which are dynamic. PiperOrigin-RevId: 281560457
* Implement unrolling of vector ops to finer-grained vector ops as a pattern.Nicolas Vasilache2019-11-202-3/+402
| | | | | | | | | This CL uses the pattern rewrite infrastructure to implement a simple VectorOps -> VectorOps legalization strategy to unroll coarse-grained vector operations into finer grained ones. The transformation is written using local pattern rewrites to allow composition with other rewrites. It proceeds by iteratively introducing fake cast ops and cleaning canonicalizing or lowering them away where appropriate. This is an example of writing transformations as compositions of local pattern rewrites that should enable us to make them significantly more declarative. PiperOrigin-RevId: 281555100
* Make type and rank explicit in mcuMemHostRegister function.Christian Sigg2019-11-191-8/+10
| | | | | | Fix registered size of indirect MemRefType kernel arguments. PiperOrigin-RevId: 281362940
* Change conversion CLI flag from -lower-to-llvm to -convert-std-to-llvmAlex Zinenko2019-11-191-1/+2
| | | | | | | | | | | | The command-line flag name `lower-to-llvm` for the pass performing dialect conversion from the Standard dialect to the LLVM dialect is misleading and inconsistent with most of the conversion passses. It leads the user to believe that there are no restrictions on what can be converted, while in fact only a subset of the Standard dialect can be converted (with operations from other dialects converted by separate passes). Use `convert-std-to-llvm` that better reflects what the pass does and is consistent with most other conversions. PiperOrigin-RevId: 281238797
* ConvertStandardToLLVM: replace assertion with graceful failureAlex Zinenko2019-11-181-2/+2
| | | | | | | | | | | | | The assertion was introduced in the early days of dialect conversion infrastructure when we had the matching function separate from the rewriting function. The infrastructure evolved to have a common matchAndRewrite function and the separate matching function was dropped without chaning the rewriting that became matchAndRewrite. This has led to assertion being triggered. Return a matchFailure instead of failing an assertion on unsupported types. Closes tensorflow/mlir#230 PiperOrigin-RevId: 281113741
* Standardize all VectorOps class names to be prefixed by Vector - NFCNicolas Vasilache2019-11-181-17/+17
| | | | | | This improves consistency and will concretely avoid collisions between VectorExtractElementOp and ExtractElementOp when they are included in the same transforms / rewrites. PiperOrigin-RevId: 281101588
* Rename CLI flags -lower-gpu-ops-to-*-ops to -convert-gpu-to-*Alex Zinenko2019-11-182-3/+2
| | | | | | | This makes the flags consistent with the naming scheme used elsewhere in the codebase for dialect conversions. PiperOrigin-RevId: 281027517
* NFC: Convert CmpIPredicate in StandardOps to use EnumAttrLei Zhang2019-11-153-13/+13
| | | | | | This turns several hand-written functions to auto-generated ones. PiperOrigin-RevId: 280684326
* Refactor the LowerVectorTransfers pass to use the RewritePattern infra - NFCNicolas Vasilache2019-11-144-4/+372
| | | | | | This is step 1/n in refactoring infrastructure along the Vector dialect to make it ready for retargetability and composable progressive lowering. PiperOrigin-RevId: 280529784
* NFC: Refactor Dialect Conversion targeting SPIR-V.Mahesh Ravishankar2019-11-146-432/+211
| | | | | | | | | | | | Refactoring the conversion from StandardOps/GPU dialect to SPIR-V dialect: 1) Move the SPIRVTypeConversion and SPIRVOpLowering class into SPIR-V dialect. 2) Add header files that expose functions to add patterns for the dialects to SPIR-V lowering, as well as a pass that does the dialect to SPIR-V lowering. 3) Make SPIRVOpLowering derive from OpLowering class. PiperOrigin-RevId: 280486871
OpenPOWER on IntegriCloud