summaryrefslogtreecommitdiffstats
path: root/mlir/test
Commit message (Collapse)AuthorAgeFilesLines
...
* [spirv] NFC: move arithmetic and logical ops to separate filesLei Zhang2019-08-233-324/+325
| | | | | | This is purely moving code around for better file organization. PiperOrigin-RevId: 265082517
* Add I32ElementsAttr to OpBaseMLIR Team2019-08-222-0/+28
| | | | PiperOrigin-RevId: 264969142
* Add iterator support to ElementsAttr and SparseElementsAttr.River Riddle2019-08-221-1/+8
| | | | | | This will allow iterating the values of a non-opaque ElementsAttr, with all of the types currently supported by DenseElementsAttr. This should help reduce the amount of specialization on DenseElementsAttr. PiperOrigin-RevId: 264968151
* [spirv] Add support for extension (de)serializationLei Zhang2019-08-222-0/+16
| | | | | | | Only a few important KHR extensions are registered to the SPIR-V dialect for now. PiperOrigin-RevId: 264939428
* Avoid overflow when lowering linalg.sliceNicolas Vasilache2019-08-221-13/+33
| | | | | | | | linalg.subview used to lower to a slice with a bounded range resulting in correct bounded accesses. However linalg.slice could still index out of bounds. This CL moves the bounding to linalg.slice. LLVM select and cmp ops gain a more idiomatic builder. PiperOrigin-RevId: 264897125
* [spirv] Add support for capability (de)serializationLei Zhang2019-08-222-0/+15
| | | | | | | This CL pulls in capabilities defined in the spec and adds support for (de)serialize capabilities of a spv.module. PiperOrigin-RevId: 264877413
* Add Positive{I32,I64}Attr and HasAnyRankOfPredLogan Chien2019-08-223-0/+74
| | | | | | | | This commit adds `PositiveI32Attr` and `PositiveI64Attr` to match positive integers but not zero nor negative integers. This commit also adds `HasAnyRankOfPred` to match tensors with the specified ranks. PiperOrigin-RevId: 264867046
* Point to spv.AccessChain when reporting spv.AccessChain errorsLei Zhang2019-08-211-7/+7
| | | | PiperOrigin-RevId: 264742130
* Remove the wrapping function in SPIR-V (de)serializationLei Zhang2019-08-2115-453/+406
| | | | | | | | | Previously Module and Function are builtinn constructs in MLIR. Due to the structural requirements we must wrap the SPIR-V module inside a Function inside a Module. Now the requirement is lifted and we can remove the wrapping function! :) PiperOrigin-RevId: 264736051
* [TableGen] Add a `StaticShapeMemRefOf` trait.MLIR Team2019-08-212-0/+11
| | | | | | The trait specifies that the `MemRefOf` has to have a static shape. PiperOrigin-RevId: 264692758
* Automated rollback of commit b9dc2e481818315f2f0d87455349f497f6118a4cRiver Riddle2019-08-211-8/+1
| | | | PiperOrigin-RevId: 264672975
* Add iterator support to ElementsAttr and SparseElementsAttr.River Riddle2019-08-211-1/+8
| | | | | | This will allow iterating the values of a non-opaque ElementsAttr, with all of the types currently supported by DenseElementsAttr. This should help reduce the amount of specialization on DenseElementsAttr. PiperOrigin-RevId: 264637293
* [spirv] Support i1 as bool typeLei Zhang2019-08-212-5/+12
| | | | PiperOrigin-RevId: 264612014
* Support variadic ops in declarative rewrite rulesLei Zhang2019-08-212-0/+162
| | | | | | | | | | | | | | | | | | | | | | | | This CL extends declarative rewrite rules to support matching and generating ops with variadic operands/results. For this, the generated `matchAndRewrite()` method for each pattern now are changed to * Use "range" types for the local variables used to store captured values (`operand_range` for operands, `ArrayRef<Value *>` for values, *Op for results). This allows us to have a unified way of handling both single values and value ranges. * Create local variables for each operand for op creation. If the operand is variadic, then a `SmallVector<Value*>` will be created to collect all values for that operand; otherwise a `Value*` will be created. * Use a collective result type builder. All result types are specified via a single parameter to the builder. We can use one result pattern to replace multiple results of the matched root op. When that happens, it will require specifying types for multiple results. Add a new collective-type builder. PiperOrigin-RevId: 264588559
* Materialize spv.constants at use sitesLei Zhang2019-08-212-90/+163
| | | | | | | | | | | | | | | In SPIR-V binary format, constants are placed at the module level and referenced by instructions inside functions using their result <id>s. To model this natively (using SSA values for result <id>s), it means we need to have implicit capturing functions. We will lose the ability to have function passes if going down that path. Instead, this CL changes to materialize constants at their use sites in deserialization. It's cheap to copy constants in MLIR given that attributes is uniqued to MLIRContext. By localizing constants into functions, we can preserve isolated functions. PiperOrigin-RevId: 264582532
* NFC: Move AffineOps dialect to the Dialect sub-directory.River Riddle2019-08-204-4/+4
| | | | PiperOrigin-RevId: 264482571
* Add spv.specConstant and spv._reference_ofLei Zhang2019-08-203-52/+174
| | | | | | | | | | | | | | | | | | | | | | | | | | | Similar to global variables, specialization constants also live in the module scope and can be referenced by instructions in functions in native SPIR-V. A direct modelling would be to allow functions in the SPIR-V dialect to implicit capture, but it means we are losing the ability to write passes for Functions. While in SPIR-V normally we want to process the module as a whole, it's not common to see multiple functions get used so we'd like to leave the door open for those cases. Therefore, similar to global variables, we introduce spv.specConstant to model three SPIR-V instructions: OpSpecConstantTrue, OpSpecConstantFalse, and OpSpecConstant. They do not return SSA value results; instead they have symbols and can only be referenced by the symbols. To use it in a function, we need to have another op spv._reference_of to turn the symbol into an SSA value. This breaks the tie and makes functions still explicit capture. Previously specialization constants were handled similarly as normal constants. That is incorrect given that specialization constant actually acts more like variable (without need to load and store). E.g., they cannot be de-duplicated like normal constants. This CL also refines various documents and comments. PiperOrigin-RevId: 264455172
* [spirv] Support (de)serialization of spv.structDenis Khalikov2019-08-201-0/+24
| | | | | | | | Support (de)serialization of spv.struct with offset decorations. Closes tensorflow/mlir#94 PiperOrigin-RevId: 264421427
* Fix build of affine load/store with empty mapDiego Caballero2019-08-201-0/+32
| | | | | | | | | | | | | | | tensorflow/mlir#58 fixed and exercised verification of load/store ops using empty affine maps. Unfortunately, it didn't exercise the creation of them. This PR addresses that aspect. It removes the assumption of AffineMap having at least one result and stores a pointer to MLIRContext as member of AffineMap. * Add empty map support to affine.store + test * Move MLIRContext to AffineMapStorage Closes tensorflow/mlir#74 PiperOrigin-RevId: 264416260
* ConvertLaunchFuncToCudaCalls: use LLVM dialect globalsAlex Zinenko2019-08-201-8/+10
| | | | | | | | | | | | This conversion has been using a stack-allocated array of i8 to store the null-terminated kernel name in order to pass it to the CUDA wrappers expecting a C string because the LLVM dialect was missing support for globals. Now that the suport is introduced, use a global instead. Refactor global string construction from GenerateCubinAccessors into a common utility function living in the LLVM namespace. PiperOrigin-RevId: 264382489
* JitRunner: support entry functions returning voidAlex Zinenko2019-08-201-1/+1
| | | | | | | | | | | | | JitRunner can use as entry points functions that produce either a single '!llvm.f32' value or a list of memrefs. Memref support is legacy and was introduced before MLIR could lower memref allocation and deallocation to malloc/free calls so as to allocate the memory externally, and is likely to be dropped in the future since it unconditionally runs affine+standard-to-llvm lowering on the module instead of accepting the LLVM dialect. CUDA runner relies on memref-based flow in the runner without actually returning anything. Introduce a runner flow to use functions that return void as entry points. PiperOrigin-RevId: 264381686
* LLVM dialect: prefix operations that correspond to intrinsics with "intr."Alex Zinenko2019-08-202-4/+4
| | | | | | | | | | LLVM intrinsics have an open name space and their names can potentially overlap with names of LLVM instructions (LLVM intrinsics are functions, not instructions). In MLIR, LLVM intrinsics are modeled as operations, so it needs to make sure their names cannot clash with the instructions. Use the "intr." prefix for intrinsics in the LLVM dialect. PiperOrigin-RevId: 264372173
* Add support for LLVM lowering of binary ops on n-D vector typesNicolas Vasilache2019-08-201-2/+27
| | | | | | This CL allows binary operations on n-D vector types to be lowered to LLVMIR by performing an (n-1)-D extractvalue, 1-D vector operation and an (n-1)-D insertvalue. PiperOrigin-RevId: 264339118
* Fix AffineExpr::simplifyAdd bugUday Bondhugula2019-08-201-0/+6
| | | | | | | | | | | | - fix missing check while simplifying an expression with floordiv to a mod - fixes issue tensorflow/mlir#82 Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Closes tensorflow/mlir#84 PiperOrigin-RevId: 264338353
* Move Linalg and VectorOps dialects to the Dialect subdir - NFCNicolas Vasilache2019-08-192-2/+2
| | | | PiperOrigin-RevId: 264277760
* Allow isolated regions to form isolated SSA name scopes in the printer.River Riddle2019-08-196-26/+71
| | | | | | | | | | | This will allow for naming values the same as existing SSA values for regions attached to operations that are isolated from above. This fits in with how the system already allows separate name scopes for sibling regions. This name shadowing can be enabled in the custom parser of operations by setting the 'enableNameShadowing' flag to true when calling 'parseRegion'. %arg = constant 10 : i32 foo.op { %arg = constant 10 : i32 } PiperOrigin-RevId: 264255999
* Add alignment support to linalg.buffer_allocNicolas Vasilache2019-08-194-58/+70
| | | | | | | | | | | | | | | | | | | | | | | This CL adds an integer attribute to linalg.buffer_alloc and lowering to LLVM. The alignment is constrained to be a positive power of 2. Lowering to LLVM produces the pattern: ``` %[[alloc:.*]] = llvm.call @malloc(%[[s]]) : (!llvm.i64) -> !llvm<"i8*"> %[[cast:.*]] = llvm.bitcast %[[alloc]] : !llvm<"i8*"> to !llvm.i64 %[[rem:.*]] = llvm.urem %[[cast]], %[[c16]] : !llvm.i64 %[[drem:.*]] = llvm.sub %[[c16]], %[[rem]] : !llvm.i64 %[[off:.*]] = llvm.urem %[[drem]], %[[c16]] : !llvm.i64 llvm.getelementptr %{{.*}}[%[[off]]] : (!llvm<"i8*">, !llvm.i64) -> !llvm<"i8*"> ``` where `ptr` is aligned on `align` by computing the address `ptr + (align - ptr % align) % align`. To allow dealloc op to still be able to free memory, additional information is needed in the buffer type. The buffer type is thus extended with an extra i8* for the base allocation address. PiperOrigin-RevId: 264244455
* Fix parsing/printing of spv.globalVariable and spv._address_ofMahesh Ravishankar2019-08-197-187/+220
| | | | | | | | | | | | Change the prining/parsing of spv.globalVariable to print the type of the variable after the ':' to be consistent with MLIR convention. The spv._address_of should print the variable type after the ':'. It was mistakenly printing the address of the return value. Add a (missing) test that should have caught that. Also move spv.globalVariable and spv._address_of tests to structure-ops.mlir. PiperOrigin-RevId: 264204686
* NFC: Move LLVMIR, SDBM, and StandardOps to the Dialect/ directory.River Riddle2019-08-194-6/+6
| | | | PiperOrigin-RevId: 264193915
* [spirv] Add spv.ReturnValueLei Zhang2019-08-193-4/+72
| | | | | | | | | This CL adds the spv.ReturnValue op and its tests. Also adds a InFunctionScope trait to make sure that the op stays inside a function. To be consistent, ModuleOnly trait is changed to InModuleScope. PiperOrigin-RevId: 264193081
* Refactor linalg lowering to LLVMNicolas Vasilache2019-08-192-84/+95
| | | | | | | | | | | | | The linalg.view type used to be lowered to a struct containing a data pointer, offset, sizes/strides information. This was problematic when passing to external functions due to ABI, struct padding and alignment issues. The linalg.view type is now lowered to LLVMIR as a *pointer* to a struct containing the data pointer, offset and sizes/strides. This simplifies the interfacing with external library functions and makes it trivial to add new functions without creating a shim that would go from a value type struct to a pointer type. The consequences are that: 1. lowering explicitly uses llvm.alloca in lieu of llvm.undef and performs the proper llvm.load/llvm.store where relevant. 2. the shim creation function `getLLVMLibraryCallDefinition` disappears. 3. views are passed by pointer, scalars are passed by value. In the future, other structs will be passed by pointer (on a per-need basis). PiperOrigin-RevId: 264183671
* Add alignment support for llvm.allocaNicolas Vasilache2019-08-183-0/+26
| | | | | | Extend the LLVM dialect AllocaOp with an alignment attribute. PiperOrigin-RevId: 264068306
* Change from llvm::make_unique to std::make_uniqueJacques Pienaar2019-08-176-7/+7
| | | | | | | | Switch to C++14 standard method as llvm::make_unique has been removed ( https://reviews.llvm.org/D66259). Also mark some targets as c++14 to ease next integrates. PiperOrigin-RevId: 263953918
* Add spirv::GlobalVariableOp that allows module level definition of variablesMahesh Ravishankar2019-08-175-62/+157
| | | | | | | | | | | | | | | | | | | | FuncOps in MLIR use explicit capture. So global variables defined in module scope need to have a symbol name and this should be used to refer to the variable within the function. This deviates from SPIR-V spec, which assigns an SSA value to variables at all scopes that can be used to refer to the variable, which requires SPIR-V functions to allow implicit capture. To handle this add a new op, spirv::GlobalVariableOp that can be used to define module scope variables. Since instructions need an SSA value, an new spirv::AddressOfOp is added to convert a symbol reference to an SSA value for use with other instructions. This also means the spirv::EntryPointOp instruction needs to change to allow initializers to be specified using symbol reference instead of SSA value The current spirv::VariableOp which returns an SSA value (as defined by SPIR-V spec) can still be used to define function-scope variables. PiperOrigin-RevId: 263951109
* [spirv] Extend spv.array with LayoutinfoDenis Khalikov2019-08-162-1/+22
| | | | | | | | Extend spv.array with Layoutinfo to support (de)serialization. Closes tensorflow/mlir#80 PiperOrigin-RevId: 263795304
* Extend vector.outerproduct with an optional 3rd argumentNicolas Vasilache2019-08-163-39/+98
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This CL adds an optional third argument to the vector.outerproduct instruction. When such a third argument is specified, it is added to the result of the outerproduct and is lowered to FMA intrinsic when the lowering supports it. In the future, we can add an attribute on the `vector.outerproduct` instruction to modify the operations for which to emit code (e.g. "+/*", "max/+", "min/+", "log/exp" ...). This CL additionally performs minor cleanups in the vector lowering and adds tests to improve coverage. This has been independently verified to result in proper fma instructions for haswell as follows. Input: ``` func @outerproduct_add(%arg0: vector<17xf32>, %arg1: vector<8xf32>, %arg2: vector<17x8xf32>) -> vector<17x8xf32> { %2 = vector.outerproduct %arg0, %arg1, %arg2 : vector<17xf32>, vector<8xf32> return %2 : vector<17x8xf32> } } ``` Command: ``` mlir-opt vector-to-llvm.mlir -vector-lower-to-llvm-dialect --disable-pass-threading | mlir-opt -lower-to-cfg -lower-to-llvm | mlir-translate --mlir-to-llvmir | opt -O3 | llc -O3 -march=x86-64 -mcpu=haswell -mattr=fma,avx2 ``` Output: ``` outerproduct_add: # @outerproduct_add # %bb.0: ... vmovaps 112(%rbp), %ymm8 vbroadcastss %xmm0, %ymm0 ... vbroadcastss 64(%rbp), %ymm15 vfmadd213ps 144(%rbp), %ymm8, %ymm0 # ymm0 = (ymm8 * ymm0) + mem ... vfmadd213ps 400(%rbp), %ymm8, %ymm9 # ymm9 = (ymm8 * ymm9) + mem ... ``` PiperOrigin-RevId: 263743359
* Add BuiltIn EnumAttr to SPIR-V dialectMahesh Ravishankar2019-08-152-0/+12
| | | | | | | | | | Generate the EnumAttr to represent BuiltIns in SPIR-V dialect. The builtIn can be specified as a StringAttr with value being the name of the builtin. Extend Decoration (de)serialization to handle BuiltIns. Also fix an error in the SPIR-V dialect generator script. PiperOrigin-RevId: 263596624
* Refactor ElementsAttr::getValue and DenseElementsAttr::getSplatValue.River Riddle2019-08-141-2/+2
| | | | | | All 'getValue' variants now require that the index is valid, queryable via 'isValidIndex'. 'getSplatValue' now requires that the attribute is a proper splat. This allows for querying these methods on DenseElementAttr with all possible value types; e.g. float, int, APInt, etc. This also allows for removing unnecessary conversions to Attribute that really want the underlying value. PiperOrigin-RevId: 263437337
* Move remaining linalg ops to ODS - NFCNicolas Vasilache2019-08-142-31/+95
| | | | | | | | This CL moves the linalg.load/range/store ops to ODS. Minor cleanups are performed. Additional invalid IR tests are added for coverage. PiperOrigin-RevId: 263432110
* Refactor linalg.view lowering to LLVM - NFCNicolas Vasilache2019-08-141-2/+2
| | | | | | This CL fuses the emission of size and stride information and makes it clearer which indexings are stepped over when querying the positions. This refactor was motivated by an index calculation bug in the stride computation. PiperOrigin-RevId: 263341610
* Move linalg.slice to ODSNicolas Vasilache2019-08-141-34/+54
| | | | PiperOrigin-RevId: 263334168
* Fix indexing issue in lowering of linalg.sliceNicolas Vasilache2019-08-131-0/+22
| | | | | | | | | | This CL fixes the stepping through operands when emitting the view sizes of linalg.slice to LLVMIR. This is now consistent with the strides emission. A relevant test is added. Fix suggested by Alex Zinenko, thanks! PiperOrigin-RevId: 263150922
* LLVM dialect: introduce fmuladd intrinsic as operationAlex Zinenko2019-08-131-0/+15
| | | | | | | | | | | This operation is important to achieve decent performance in computational kernels. In LLVM, it is implemented as an intrinsic (through function declaration and function call). Thanks to MLIR's extendable set of operations, it does not have to differentiate between built-ins and intrinsics, so fmuladd is introduced as a general type-polymorphic operation. Custom printing and parsing will be added later. PiperOrigin-RevId: 263106305
* GenerateCubinAccessors: use LLVM dialect constantsAlex Zinenko2019-08-131-26/+9
| | | | | | | | | | | | The GenerateCubinAccessors was generating functions that fill dynamically-allocated memory with the binary constant of a CUBIN attached as a stirng attribute to the GPU kernel. This approach was taken to circumvent the missing support for global constants in the LLVM dialect (and MLIR in general). Global constants were recently added to the LLVM dialect. Change the GenerateCubinAccessors pass to emit a global constant array of characters and a function that returns a pointer to the first character in the array. PiperOrigin-RevId: 263092052
* Express ownership transfer in PassManager API through std::unique_ptr (NFC)Mehdi Amini2019-08-126-13/+17
| | | | | | | | | | | | | | Since raw pointers are always passed around for IR construct without implying any ownership transfer, it can be error prone to have implicit ownership transferred the same way. For example this code can seem harmless: Pass *pass = .... pm.addPass(pass); pm.addPass(pass); pm.run(module); PiperOrigin-RevId: 263053082
* LLVM dialect: introduce llvm.addressof to access globalsAlex Zinenko2019-08-122-0/+61
| | | | | | | | | | | This instruction is a local counterpart of llvm.global that takes a symbol reference to a global and produces an SSA value containing the pointer to it. Used in combination, these two operations allow one to use globals with other operations expecting SSA values. At a cost of IR indirection, we make sure the functions don't implicitly capture the surrounding SSA values and remain suitable for parallel processing. PiperOrigin-RevId: 262908622
* Add lowering of vector dialect to LLVM dialect.Nicolas Vasilache2019-08-121-0/+33
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This CL is step 3/n towards building a simple, programmable and portable vector abstraction in MLIR that can go all the way down to generating assembly vector code via LLVM's opt and llc tools. This CL adds support for converting MLIR n-D vector types to (n-1)-D arrays of 1-D LLVM vectors and a conversion VectorToLLVM that lowers the `vector.extractelement` and `vector.outerproduct` instructions to the proper mix of `llvm.vectorshuffle`, `llvm.extractelement` and `llvm.mulf`. This has been independently verified to produce proper avx2 code. Input: ``` func @vec_1d(%arg0: vector<4xf32>, %arg1: vector<8xf32>) -> vector<8xf32> { %2 = vector.outerproduct %arg0, %arg1 : vector<4xf32>, vector<8xf32> %3 = vector.extractelement %2[0 : i32]: vector<4x8xf32> return %3 : vector<8xf32> } ``` Command: ``` mlir-opt vector-to-llvm.mlir -vector-lower-to-llvm-dialect --disable-pass-threading | mlir-opt -lower-to-cfg -lower-to-llvm | mlir-translate --mlir-to-llvmir | opt -O3 | llc -O3 -march=x86-64 -mcpu=haswell -mattr=fma,avx2 ``` Output: ``` vec_1d: # @vec_1d # %bb.0: vbroadcastss %xmm0, %ymm0 vmulps %ymm1, %ymm0, %ymm0 retq ``` PiperOrigin-RevId: 262895929
* NFC: Standardize the terminology used for parent ops/regions/etc.River Riddle2019-08-093-3/+3
| | | | | | There are currently several different terms used to refer to a parent IR unit in 'get' methods: getParent/getEnclosing/getContaining. This cl standardizes all of these methods to use 'getParent*'. PiperOrigin-RevId: 262680287
* NFC: Update usages of OwningRewritePatternList to pass by & instead of &&.River Riddle2019-08-091-4/+3
| | | | | | This will allow for reusing the same pattern list, which may be costly to continually reconstruct, on multiple invocations. PiperOrigin-RevId: 262664599
* LLVM dialect and translation: support global stringsAlex Zinenko2019-08-092-1/+23
| | | | | | | | | | | Unlike regular constant values, strings must be placed in some memory and referred to through a pointer to that memory. Until now, they were not supported in function-local constant declarations with `llvm.constant`. Introduce support for global strings using `llvm.global`, which would translate them into global arrays in LLVM IR and thus make sure they have some memory allocated for storage. PiperOrigin-RevId: 262569316
OpenPOWER on IntegriCloud