summaryrefslogtreecommitdiffstats
path: root/mlir/lib/Parser/Lexer.cpp
Commit message (Collapse)AuthorAgeFilesLines
* Adjust License.txt file to use the LLVM licenseMehdi Amini2019-12-231-13/+4
| | | | PiperOrigin-RevId: 286906740
* NFC: Fix mismatches between LangRef.md and actual parser implementation.River Riddle2019-12-041-1/+1
| | | | PiperOrigin-RevId: 283805832
* Restructure comment lexing to not recurse.River Riddle2019-11-111-99/+92
| | | | | | In some files that have large amounts of comments, this can lead to a stack overflow. PiperOrigin-RevId: 279867330
* Fix minor spelling tweaks (NFC)Kazuaki Ishizaki2019-10-201-2/+2
| | | | | | Closes tensorflow/mlir#177 PiperOrigin-RevId: 275692653
* Add support for parsing/printing non bare-identifier SymbolRefs.River Riddle2019-10-081-3/+12
| | | | | | The restriction that symbols can only have identifier names is arbitrary, and artificially limits the names that a symbol may have. This change adds support for parsing and printing symbols that don't fit in the 'bare-identifier' grammar by printing the reference in quotes, e.g. @"0_my_reference" can now be used as a symbol name. PiperOrigin-RevId: 273644768
* Fix a number of Clang-Tidy warnings.Christian Sigg2019-09-231-1/+1
| | | | PiperOrigin-RevId: 270632324
* NFC: Add doc for id-punctAlina Sbirlea2019-08-231-0/+1
| | | | PiperOrigin-RevId: 265190168
* Lexer: NFC: sort helper methods alphabeticallyAlex Zinenko2019-08-081-93/+93
| | | | | | | | Lexer methods were added progressively as implementation advanced. The rest of MLIR now tends to sort methods alphabetically for better discoverability in absence of tooling. Sort the lexer methods as well. PiperOrigin-RevId: 262406992
* Introduce support for variadic function signatures for the LLVM dialectAlex Zinenko2019-08-081-0/+16
| | | | | | | | | | | | | LLVM function type has first-class support for variadic functions. In the current lowering pipeline, it is emulated using an attribute on functions of standard function type. In LLVMFuncOp that has LLVM function type, this can be modeled directly. Introduce parsing support for variadic arguments to the function and use it to support variadic function declarations in LLVMFuncOp. Function definitions are currently not supported as that would require modeling va_start/va_end LLVM intrinsics in the dialect and we don't yet have a consistent story for LLVM intrinsics. PiperOrigin-RevId: 262372651
* Update style/clang-format (NFC).Jacques Pienaar2019-07-221-27/+48
| | | | | | Update to be consistent & so that future save + clang-format workflows don't introduce extra changes. PiperOrigin-RevId: 259361174
* Rename FunctionAttr to SymbolRefAttr.River Riddle2019-07-121-1/+1
| | | | | | This allows for the attribute to hold symbolic references to other operations than FuncOp. This also allows for removing the dependence on FuncOp from the base Builder. PiperOrigin-RevId: 257650017
* Move the emitError/Warning/Remark utility methods out of MLIRContext and ↵River Riddle2019-06-251-2/+2
| | | | | | | | into the mlir namespace. Now that Locations are attributes, they have direct access to the MLIR context. This allows for simplifying error emission by removing unnecessary context lookups. PiperOrigin-RevId: 255112791
* Replace usages of 'UniquedFilename' with 'Identifier' and remove it. ↵River Riddle2019-06-191-2/+2
| | | | | | Identifier already contains all of the necessary functionality/verification, so having a separate class for filenames is unnecessary. PiperOrigin-RevId: 253855505
* Refactor the support for AffineMap and IntegerSet aliases in the parser ↵River Riddle2019-05-101-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | into more general support for attribute aliases. `#` alias `=` attribute-value This also allows for dialects to define aliases for attributes in the AsmPrinter. The printer supports two types of attribute aliases, 'direct' and 'kind'. * Direct aliases are synonymous with the current support for type aliases, i.e. this maps an alias to a specific instance of an attribute. // A direct alias ("foo_str") for the string attribute "foo". #foo_str = "foo" * Kind aliases generates unique names for all instances of a given attribute kind. The generated aliases are of the form: `alias[0-9]+`. // A kind alias ("strattr") for all string attributes could generate. #strattr0 = "foo" #strattr1 = "bar" ... #strattrN = "baz" -- PiperOrigin-RevId: 246851916
* Introduce a new API for emitting diagnostics with Diagnostic and ↵River Riddle2019-05-061-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | InFlightDiagnostic. The Diagnostic class contains all of the information necessary to report a diagnostic to the DiagnosticEngine. It should generally not be constructed directly, and instead used transitively via InFlightDiagnostic. A diagnostic is currently comprised of several different elements: * A severity level. * A source Location. * A list of DiagnosticArguments that help compose and comprise the output message. * A DiagnosticArgument represents any value that may be part of the diagnostic, e.g. string, integer, Type, Attribute, etc. * Arguments can be added to the diagnostic via the stream(<<) operator. * (In a future cl) A list of attached notes. * These are in the form of other diagnostics that provide supplemental information to the main diagnostic, but do not have context on their own. The InFlightDiagnostic class represents an RAII wrapper around a Diagnostic that is set to be reported with the diagnostic engine. This allows for the user to modify a diagnostic that is inflight. The internally wrapped diagnostic can be reported directly or automatically upon destruction. These classes allow for more natural composition of diagnostics by removing the restriction that the message of a diagnostic is comprised of a single Twine. They should also allow for nice incremental improvements to the diagnostics experience in the future, e.g. formatv style diagnostics. Simple Example: emitError(loc, "integer bitwidth is limited to " + Twine(IntegerType::kMaxWidth) + " bits"); emitError(loc) << "integer bitwidth is limited to " << IntegerType::kMaxWidth << " bits"; -- PiperOrigin-RevId: 246526439
* Allow function names to have a leading underscore. This matches what is ↵River Riddle2019-03-291-3/+5
| | | | | | already defined in the spec, but not supported in the implementation. PiperOrigin-RevId: 235823663
* Validate the names of attribute, dialect, and functions during verification. ↵River Riddle2019-03-291-1/+2
| | | | | | This essentially enforces the parsing rules upon their names. PiperOrigin-RevId: 235818842
* Disallow hexadecimal literals in type declarationsAlex Zinenko2019-03-291-4/+5
| | | | | | | | | | | | | | | | | | | Existing IR syntax is ambiguous in type declarations in presence of zero sizes. In particular, `0x1` in the type size can be interpreted as either a hexadecimal literal corresponding to 1, or as two distinct decimal literals separated by an `x` for sizes. Furthermore, the shape `<0xi32>` fails lexing because it is expected to be an integer literal. Fix the lexer to treat `0xi32` as an integer literal `0` followed by a bare identifier `xi32` (look one character ahead and early return instead of erroring out). Disallow hexadecimal literals in type declarations and forcibly split the token into multiple parts while parsing the type. Note that the splitting trick has been already present to separate the element type from the preceding `x` character. PiperOrigin-RevId: 232880373
* Minor fix to the lexer whitespace loop.Stella Laurenzo2019-03-291-1/+4
| | | | | | | | | | | | Nothing in the loop can (legally) cause curPtr -> nullptr. And if it did, we would null dereference right below anyway. This loop still reads funny to me but doesn't make me stare at it and wonder what I am missing anymore. -- PiperOrigin-RevId: 232062076
* Follow up from previous change to avoid setting tokStart 2x.Jacques Pienaar2019-03-291-3/+1
| | | | PiperOrigin-RevId: 228903980
* Skip over whitespace using loop. NFC.Jacques Pienaar2019-03-291-7/+16
| | | | | | Else we can stack overflow on a long sequence of whitespace. PiperOrigin-RevId: 228893517
* Add parser support for named type aliases.River Riddle2019-03-291-1/+2
| | | | | | | | | | | | | | | | | | | | Alias identifiers can be used in the place of the types that they alias, and are defined as: type-alias-def ::= '!' alias-name '=' 'type' type type-alias ::= '!' alias-name Example: !avx.m128 = type vector<4 x f32> ... "foo"(%x) : vector<4 x f32> -> () // becomes: "foo"(%x) : !avx.m128 -> () PiperOrigin-RevId: 228271372
* Implement initial support for dialect specific types.River Riddle2019-03-291-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | Dialect specific types are registered similarly to operations, i.e. registerType<...> within the dialect. Unlike operations, there is no notion of a "verbose" type, that is *all* types must be registered to a dialect. Casting support(isa/dyn_cast/etc.) is implemented by reserving a range of type kinds in the top level Type class as opposed to string comparison like operations. To support derived types a few hooks need to be implemented: In the concrete type class: - static char typeID; * A unique identifier for the type used during registration. In the Dialect: - typeParseHook and typePrintHook must be implemented to provide parser support. The syntax for dialect extended types is as follows: dialect-type: '!' dialect-namespace '<' '"' type-specific-data '"' '>' The 'type-specific-data' is information used to identify different types within the dialect, e.g: - !tf<"variant"> // Tensor Flow Variant Type - !tf<"string"> // Tensor Flow String Type TensorFlow/TensorFlowControl types are now implemented as dialect specific types as a proof of concept. PiperOrigin-RevId: 227580052
* Introduce ^ as a basic block sigil, eliminating an ambiguity on the MLIRChris Lattner2019-03-291-1/+9
| | | | | | syntax. PiperOrigin-RevId: 227234174
* Return bool from all emitError methods similar to Operation::emitOpErrorSmit Hinsu2019-03-291-2/+2
| | | | | | | | | | | | | This simplifies call-sites returning true after emitting an error. After the conversion, dropped braces around single statement blocks as that seems more common. Also, switched to emitError method instead of emitting Error kind using the emitDiagnostic method. TESTED with existing unit tests PiperOrigin-RevId: 224527868
* Implement value type abstraction for locations.River Riddle2019-03-291-1/+1
| | | | | | Value type abstraction for locations differ from others in that a Location can NOT be null. NOTE: dyn_cast returns an Optional<T>. PiperOrigin-RevId: 220682078
* Change sigil for integer set: @@ -> #Uday Bondhugula2019-03-291-18/+0
| | | | PiperOrigin-RevId: 218786684
* Fix some leak and crash found via fuzzing.Jacques Pienaar2019-03-291-1/+1
| | | | | | Tried adding a fuzzer target (cl/216378253) and ran into a few problems, and fixing two of these. PiperOrigin-RevId: 216425403
* Change unranked tensor syntax from tensor<??f32> to tensor<*xf32> perChris Lattner2019-03-291-5/+0
| | | | | | discussion on the list. PiperOrigin-RevId: 212838226
* Change SourgeMgr to const reference in Parser/Lexer.Jacques Pienaar2019-03-291-1/+1
| | | | | | SourceMgr is not be mutated during parsing/lexing. PiperOrigin-RevId: 212145759
* Continue revising diagnostic handling to simplify and generalize it, and ↵Chris Lattner2019-03-291-5/+19
| | | | | | | | | | | | | | | | | | | improve related infra. - Add a new -verify mode to the mlir-opt tool that allows writing test cases for optimization and other passes that produce diagnostics. - Refactor existing the -check-parser-errors flag to mlir-opt into a new -split-input-file option which is orthogonal to -verify. - Eliminate the special error hook the parser maintained and use the standard MLIRContext's one instead. - Enhance the default MLIRContext error reporter to print file/line/col of errors when it is available. - Add new createChecked() methods to the builder that create ops and invoke the verify hook on them, use this to detected unhandled code in the RaiseControlFlow pass. - Teach mlir-opt about expected-error @+, it previously only worked with @- PiperOrigin-RevId: 211305770
* Escape and unescape strings in the parser and printer so they can roundtrip,Chris Lattner2019-03-291-3/+8
| | | | | | | | | | | print floating point in a structured form that we know can round trip, enumerate attributes in the visitor so we print affine mapping attributes symbolically (the majority of the testcase updates). We still have an issue where the hexadecimal floating point syntax is reparsed as an integer, but that can evolve in subsequent patches. PiperOrigin-RevId: 208828876
* [mlir] Allow C-style escapes in LexerJames Molloy2019-03-291-0/+6
| | | | | | This patch passes the raw, unescaped value through to the rest of the stack. Partial escaping is a total pain to deal with, so we either need to implement escaping properly (ideally using a third party library like absl, I don't think LLVM has one that can handle the proper gamut of escape codes) or don't escape. I chose the latter for this patch. PiperOrigin-RevId: 208608945
* Support for affine integer setsUday Bondhugula2019-03-291-1/+21
| | | | | | | | | | | | | - introduce affine integer sets into the IR - parse and print affine integer sets (both inline or outlined) similar to affine maps - use integer set for IfStmt's conditional, and implement parsing of IfStmt's conditional - fixed an affine expr paren omission bug while one this. TODO: parse/represent/print MLValue operands to affine integer set references. PiperOrigin-RevId: 207779408
* Add parsing for floating point attributes.Jacques Pienaar2019-03-291-2/+18
| | | | | | This is doing it in a suboptimal manner by recombining [integer period literal] into a string literal and parsing that via to_float. PiperOrigin-RevId: 206855106
* Add . to bare-id to allow custom ops such as tf.addJacques Pienaar2019-03-291-3/+3
| | | | PiperOrigin-RevId: 206840659
* Prepare for implementation of TensorFlow passes:Chris Lattner2019-03-291-1/+5
| | | | | | | | | | - Sketch out a TensorFlow/IR directory that will hold op definitions and common TF support logic. We will eventually have TensorFlow/TF2HLO, TensorFlow/Grappler, TensorFlow/TFLite, etc. - Add sketches of a Switch/Merge op definition, including some missing stuff like the TwoResults trait. Add a skeleton of a pass to raise this form. - Beef up the Pass/FunctionPass definitions slightly, moving the common code out of LoopUnroll.cpp into a new IR/Pass.cpp file. - Switch ConvertToCFG.cpp to be a ModulePass. - Allow _ to start bare identifiers, since this is important for TF attributes. PiperOrigin-RevId: 206502517
* Add tf_control type and allow $ in bare-id.Jacques Pienaar2019-03-291-4/+4
| | | | | | | * Add tf_control as primitive type; * Allow $ in bare-id to allow attributes with $ (to make it trivially to mangle a TF attribute); PiperOrigin-RevId: 206342642
* Implement custom parser support for operations, enhance dim/addf to use it, ↵Chris Lattner2019-03-291-2/+4
| | | | | | | | | | and add a new load op. This regresses parser error recovery in some cases (in invalid.mlir) which I'll consider in a follow-up patch. The important thing in this patch is that the parse methods in StandardOps.cpp are nice and simple. PiperOrigin-RevId: 206023308
* Switch the comment syntax from ; to // comments as discussed on Friday. ThereChris Lattner2019-03-291-1/+9
| | | | | | | is no strong reason to prefer one or the other, but // is nice for consistency given the rest of the compiler is written in C++. PiperOrigin-RevId: 204628476
* Remove const reference to errorReporter.Jacques Pienaar2019-03-291-2/+1
| | | | | | Fixes use-after-free ASAN failure. PiperOrigin-RevId: 204177796
* Add basic lexing and parsing support for SSA operands and definitions. ThisChris Lattner2019-03-291-7/+25
| | | | | | | isn't actually constructing IR objects yet, it is eating the tokens and discarding them. PiperOrigin-RevId: 203616265
* Parsing support for affine maps and affine expressionsUday Bondhugula2019-03-291-1/+1
| | | | | | | | A recursive descent parser for affine maps/expressions with operator precedence and associativity. (While on this, sketch out uniqui'ing functionality for affine maps and affine binary op expressions (partly).) PiperOrigin-RevId: 203222063
* Enhance the type system to support arbitrary precision integers, which areChris Lattner2019-03-291-0/+10
| | | | | | | | | important for low-bitwidth inference cases and hardware synthesis targets. Rename 'int' to 'affineint' to avoid confusion between "the integers" and "the int type". PiperOrigin-RevId: 202751508
* [WIP] Sketching IR and parsing support for affine maps, affine expressionsUday Bondhugula2019-03-291-0/+6
| | | | | | | | | | | Run test case: $ mlir-opt test/IR/parser-affine-map.mlir test/IR/parser-affine-map.mlir:3:30: error: expect '(' at start of map range #hello_world2 (i, j) [s0] -> i+s0, j) ^ PiperOrigin-RevId: 202736856
* Refactor information about tokens out into a new TokenKinds.def file. Use thisChris Lattner2019-03-291-20/+5
| | | | | | | | to share code a bit more, and fixes a diagnostic bug Uday pointed out where parseCommaSeparatedList would print the wrong diagnostic when the end signifier was not a ). PiperOrigin-RevId: 202676858
* Sketch out parser/IR support for OperationInst, and a new Instruction baseChris Lattner2019-03-291-0/+30
| | | | | | | | | | class. Introduce an Identifier class to MLIRContext to represent uniqued identifiers, introduce string literal support to the lexer, introducing parser and printer support etc. PiperOrigin-RevId: 202592007
* Add some scaffolding for parsing affine maps:MLIR Team2019-03-291-0/+31
| | | | | | | | - parsing affine map identifiers - place-holder classes for AffineMap - module contains a list of affine maps (defined at the top level). PiperOrigin-RevId: 202336919
* Change Lexer and Parser to take diagnostic reporter function.Jacques Pienaar2019-03-291-5/+5
| | | | | | Add diagnostic reporter function to lexer/parser and use that from mlir-opt to report errors instead of having the lexer/parser print the errors. PiperOrigin-RevId: 201892004
* Add the unconditional branch instruction, improve diagnostics for blockChris Lattner2019-03-291-5/+6
| | | | | | references. PiperOrigin-RevId: 201872745
OpenPOWER on IntegriCloud