summaryrefslogtreecommitdiffstats
path: root/llvm/utils/lit
Commit message (Collapse)AuthorAgeFilesLines
* [AVR] Whitelist the avrlit config environment variablesDylan McKay2016-12-151-1/+1
| | | | | | This allows us to use `lit` to run on-target execution tests. llvm-svn: 289769
* Revert "[AVR] Add the very first on-target test"Renato Golin2016-12-141-1/+1
| | | | | | | This reverts commit r289648, as it's an execution test and relies on the emulator/dispatcher being available on all builders. llvm-svn: 289651
* [AVR] Add the very first on-target testDylan McKay2016-12-141-1/+1
| | | | | | This test runs on actual AVR hardware. llvm-svn: 289648
* [lit] Support custom parsers in parseIntegratedTestScriptEric Fiselier2016-12-054-50/+298
| | | | | | | | | | | | | | | | | | | | | | | Summary: Libc++ frequently has the need to parse more than just the builtin *test keywords* (`RUN`, `REQUIRES`, `XFAIL`, ect). For example libc++ currently needs a new keyword `MODULES-DEFINES: macro list...`. Instead of re-implementing the script parsing in libc++ this patch allows `parseIntegratedTestScript` to take custom parsers. This patch introduces a new class `IntegratedTestKeywordParser` which implements the logic to parse/process a test keyword. Parsing of various keyword "kinds" are supported out of the box, including 'TAG', 'COMMAND', and 'LIST', which parse keywords such as `END.`, `RUN:` and `XFAIL:` respectively. As an example after this change libc++ can implement the `MODULES-DEFINES` simply using: ``` mparser = IntegratedTestKeywordParser('MODULES-DEFINES:', ParserKind.LIST) parseIntegratedTestScript(test, additional_parsers=[mparser]) macro_list = mparser.getValue() ``` Reviewers: ddunbar, modocache, rnk, danalbert, jroelofs Subscribers: mgrang, llvm-commits, cfe-commits Differential Revision: https://reviews.llvm.org/D27005 llvm-svn: 288694
* Recommit r287403 (reverted in r287804): [lit] When setting SDKROOT on ↵Kuba Mracek2016-12-011-1/+1
| | | | | | | | Darwin, use '--sdk macosx' to find the right SDK path. This shouls now be safe and not break any more bots. It's strictly better to use '--sdk macosx', otherwise xcrun can return weird things for example when you have Command Line Tools or the SDK installed into '/'. llvm-svn: 288385
* Revert "[lit] When setting SDKROOT on Darwin, use '--sdk macosx' to find the ↵Vedant Kumar2016-11-231-1/+1
| | | | | | | | | right SDK path." This reverts commit r287403. It breaks an internal asan bot. According to Kuba, a fix is up for review here: https://reviews.llvm.org/D26929 llvm-svn: 287804
* [lit] When setting SDKROOT on Darwin, use '--sdk macosx' to find the right ↵Kuba Mracek2016-11-181-1/+1
| | | | | | | | SDK path. This will make sure that we find an actual path in case you have Command Line Tools installed. llvm-svn: 287403
* [lit] Print negative exit codes on Windows in hexReid Kleckner2016-11-071-2/+8
| | | | | | | Negative exit codes are usually exceptions. They're easier to recognize in hex. Compare -1073741502 to 0xc0000142. llvm-svn: 286150
* [lit] Remove TODOBrian Gesiak2016-11-032-175/+3
| | | | | | | | | | | | | | | | | | | | | | Summary: Instead of keeping track of TODOs for lit in a file checked into source control, use LLVM's bug tracker. The TODOs have been migrated to the following bugs: * https://llvm.org/bugs/show_bug.cgi?id=30666 * https://llvm.org/bugs/show_bug.cgi?id=30667 * https://llvm.org/bugs/show_bug.cgi?id=30668 * https://llvm.org/bugs/show_bug.cgi?id=30669 * https://llvm.org/bugs/show_bug.cgi?id=30670 * https://llvm.org/bugs/show_bug.cgi?id=30671 Reviewers: ddunbar, beanz, echristo, delcypher Subscribers: llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D25496 llvm-svn: 285973
* [lit] Work around Windows MSys command line tokenization bugReid Kleckner2016-10-261-0/+61
| | | | | | | | | | | | | | Summary: This will allow us to revert LLD r284768, which added spaces to get MSys echo to print what we want. Reviewers: ruiu, inglorion, rafael Subscribers: modocache, llvm-commits Differential Revision: https://reviews.llvm.org/D26009 llvm-svn: 285237
* [lit] Add more testing instructions to READMEBrian Gesiak2016-10-221-0/+15
| | | | | | | | | | | | | | | Summary: r283710 introduced two regressions, one to llvm-lit, and the other to lit executables that were installed via setuptools. Add instructions on how to test for these regressions in the future. Reviewers: ddunbar, delcypher, beanz, chapuni, cmatthews, echristo Subscribers: llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D25459 llvm-svn: 284919
* [lit] Fix test shtest-timeout.py for modern outputMichal Gorny2016-10-121-10/+9
| | | | | | | | | | Update the CHECK lines in the shtest-timeout.py lit test to account for the current output. The output has been changed in r271610 without adjusting the tests. Differential Revision: https://reviews.llvm.org/D25236 llvm-svn: 284057
* [lit] Fix FormatError on individual test timeoutMichal Gorny2016-10-121-1/+1
| | | | | | Differential Revision: https://reviews.llvm.org/D25195 llvm-svn: 284056
* [lit] Run unit tests as part of lit test suiteBrian Gesiak2016-10-122-108/+112
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: The Python file `utils/lit/lit/ShUtil.py` contains: 1. Logic used by lit itself 2. A set of unit tests for that logic, which can be run by invoking `python utils/lit/lit/ShUtil.py` Move these unit tests to a `tests/unit` subdirectory of lit, and run the tests as part of lit's test suite. This ensures that, should the lit test suite be included in LLVM's own regression test suite, these unit tests will also be run. (Instructions on how to run lit's test suite can be found in `utils/lit/README.txt`.) Reviewers: ddunbar, echristo, delcypher, beanz Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D25411 llvm-svn: 283968
* Fix issue which cases lit installed with setup.py to not resolve mainChris Matthews2016-10-101-0/+2
| | | | llvm-svn: 283818
* [lit] Remove (or allow specific) unused importsBrian Gesiak2016-10-104-10/+12
| | | | | | | | | | | | | | | | | | | | | Summary: Using Python linter flake8 on the utils/lit reveals several linter warnings designated "F401: Unused import". Fix or silence these warnings. Some of these unused imports are legitimate, while some are part of lit's API. For example, users of lit expect to be able to access `lit.formats.ShTest` in their `lit.cfg`, despite the module hierarchy for that symbol actually being `lit.formats.shtest.ShTest`. To silence linter errors for these lines, include a "noqa" directive. Reviewers: echristo, delcypher, beanz, ddunbar Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D25407 llvm-svn: 283710
* [lit] Remove unused TestingProgressDisplay attrBrian Gesiak2016-10-101-1/+0
| | | | | | | | | | | | | | Summary: `TestingProgressDisplay` initializes its `current` attribute to `None`, but never reads or writes the value again. Remove it. Reviewers: echristo, delcypher, beanz, ddunbar Subscribers: llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D25415 llvm-svn: 283709
* [lit] Fix undefined symbol ArgumentErrorBrian Gesiak2016-10-101-2/+2
| | | | | | | | | | | | | | | | | | | Summary: `ArgumentError` is not defined by the Python standard library. Executing this line of code would throw a exception, but not the intended one. It would throw a `NameError` exception, since `ArgumentError` is undefined. Use `ValueError` instead, which is defined by the Python standard library. Reviewers: echristo, delcypher, beanz, ddunbar Subscribers: llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D25410 llvm-svn: 283708
* [lit] Remove semicolons in Python codeBrian Gesiak2016-10-102-3/+3
| | | | | | | | | | | | | | | Summary: Semicolons aren't necessary as statement terminators in Python, and each of these uses are superfluous as they appear at the end of a line. The convention is to not use semicolons where not needed, so remove them. Reviewers: echristo, delcypher, beanz, ddunbar Subscribers: llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D25409 llvm-svn: 283707
* [lit] Remove unused variable in googletest formatBrian Gesiak2016-10-101-1/+0
| | | | | | | | | | | | Summary: `prefix` is written to but never read. Reviewers: echristo, delcypher, beanz, ddunbar Subscribers: llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D25408 llvm-svn: 283706
* [lit] Remove Python 2.6 and below exec workaroundBrian Gesiak2016-10-101-12/+7
| | | | | | | | | | | | | | Summary: The minimum version of Python required to run LLVM's test suite is 2.7. Remove a workaround for older Python versions. Reviewers: echristo, delcypher, beanz, ddunbar Subscribers: llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D25400 llvm-svn: 283705
* [lit] Remove workaround for Python 2.5Brian Gesiak2016-10-041-8/+1
| | | | | | | | | | | | | | Summary: The minimum version of Python necessary to run the LLVM test suite is 2.7. Code to work around Python 2.5 and lower isn't necessary. Reviewers: ddunbar, echristo, delcypher, beanz Subscribers: llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D25209 llvm-svn: 283169
* [lit] Use argparse instead of optparseChris Bieneman2016-10-032-46/+45
| | | | | | | | | | | | | | | | | | | | | | | | Summary: optparse is deprecated in Python 2.7, which is the minimum version of Python required to run the LLVM test suite. Replace its usage in lit with argparse, optparse's 2.7 replacement module. argparse has several benefits over optparse, but this commit does not make use of those benefits yet. Instead, it simply uses the new API, and attempts to keep the number of changes to a minimum. Confirmed that lit's test suite, as well as LLVM's regression test suite, still pass with these changes. Patch By Brian Gesiak! Reviewers: ddunbar, echristo, beanz, delcypher Subscribers: llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D25173 llvm-svn: 283152
* [lit] Throw in unimplemented method (NFC)Chris Bieneman2016-10-031-1/+1
| | | | | | | | | | | | | | | | | | Summary: lit's `OneCommandFileTest` class implements an abstract method that raises if called. However, it raises by referencing an undefined symbol. Instead, raise explicitly by throwing a `NotImplementedError`. This is clearer, and appeases Python linters. Patch By Brian Gesiak! Reviewers: ddunbar, echristo, beanz Subscribers: llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D25170 llvm-svn: 283090
* [lit] Remove unused imports (NFC)Chris Bieneman2016-10-034-5/+6
| | | | | | | | | | | | Reviewers: ddunbar, echristo, beanz Patch by Brian Gesiak! Subscribers: llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D25169 llvm-svn: 283089
* [lit] Compare to None using identity, not equalityChris Bieneman2016-10-032-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | Summary: In Python, `None` is a singleton, so checking whether a variable is `None` may be done with `is` or `is not`. This has a slight advantage over equiality comparisons `== None` and `!= None`, since `__eq__` may be overridden in Python to produce sometimes unexpected results. Using `is None` and `is not None` is also recommended practice in https://www.python.org/dev/peps/pep-0008: > Comparisons to singletons like `None` should always be done with `is` or > `is not`, never the equality operators. Patch by Brian Gesiak! Reviewers: ddunbar, echristo, beanz Subscribers: llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D25168 llvm-svn: 283088
* [lit] Add instructions to run lit's test suiteDaniel Dunbar2016-09-271-0/+16
| | | | | | | | - Patch by Brian Gesiak. - https://reviews.llvm.org/D24968 llvm-svn: 282525
* [lit] Fix refacto introduced by rL282479.Daniel Dunbar2016-09-271-1/+3
| | | | llvm-svn: 282501
* Trying to fix lldb build breakage probably caused by rL282452Dimitar Vlahovski2016-09-271-11/+11
| | | | llvm-svn: 282479
* [lit] Add a --max-failures option.Daniel Dunbar2016-09-264-3/+41
| | | | | | | | | - This is primarily useful as a "fail fast" mode for lit, where it will stop running tests after the first failure. - Patch by Max Moiseev. llvm-svn: 282452
* [lit] Downgrade error to warning on gtest crashes during discovery.Ahmed Bougacha2016-09-051-2/+2
| | | | | | | | | | Lots of unittests started failing under asan after r280455. It seems they've been failing for a long time, but lit silently ignored them. Downgrade the error so we can figure out what is going on. Filed http://llvm.org/PR30285. llvm-svn: 280674
* lit/util.py: Another fix for py3.NAKAMURA Takumi2016-09-051-0/+2
| | | | | | 'str' object has no attribute 'decode'. llvm-svn: 280641
* Make lit/util.py py3-compatible.NAKAMURA Takumi2016-09-031-1/+1
| | | | llvm-svn: 280579
* lit: print process output, if getting the list of google-tests failed.Ivan Krasin2016-09-022-6/+11
| | | | | | | | | | | | | Summary: This is a follow up to r280455, where a check for the process exit code was introduced. Some ASAN bots throw this error now, but it's impossible to understand what's wrong with them, and the issue is not reproducible. Reviewers: vitalybuka Differential Revision: https://reviews.llvm.org/D24210 llvm-svn: 280550
* Quick fix to make LIT_PRESERVES_TMP work againReid Kleckner2016-09-021-1/+2
| | | | llvm-svn: 280502
* [lit] Clean up temporary files created by testsReid Kleckner2016-09-022-11/+28
| | | | | | | | | | | | Do this by creating a temp directory in the normal system temp directory, and cleaning it up on exit. It is still possible for this temp directory to leak if Python exits abnormally, but this is probably good enough for now. Fixes PR18335 llvm-svn: 280501
* [lit] Fail testing if a googletest executable crashes during test discoveryGreg Parker2016-09-022-2/+6
| | | | | | | | | googletest formatted tests are discovered by running the test executable. Previously testing would silently succeed if the test executable crashed during the discovery process. Now testing fails with "error: unable to discover google-tests ..." if the test executable exits with a non-zero status. llvm-svn: 280455
* [lit] Use multiprocessing by default on WindowsReid Kleckner2016-09-011-6/+3
| | | | | | | | | | | | | | | | Apparently nobody evaluated multiprocessing on Windows since Daniel enabled multiprocessing on Unix in r193279. It works so far as I can tell. Today this is worth about an 8x speedup (631.29s to 73.25s) on my 24 core Windows machine. Hopefully this will improve Windows buildbot cycle time, where currently it takes more time to run check-all than it does to self-host with assertions enabled: http://lab.llvm.org:8011/builders/clang-x86-windows-msvc2015/builds/20 build stage 2 ninja all ( 28 mins, 22 secs ) ninja check 2 stage 2 ( 37 mins, 38 secs ) llvm-svn: 280382
* [lit] Use full config path in diagnostics.Daniel Dunbar2016-07-211-1/+1
| | | | | | | | | - This allows tools like emacs to automatically find the config file path when you step through errors. - Patch by Dave Abrahams. llvm-svn: 276357
* [lit] Bump version number.Daniel Dunbar2016-07-211-2/+2
| | | | llvm-svn: 276353
* [lit] Add SANITIZER_IGNORE_CVE_2016_2143 to pass_vars.Marcin Koscielnicki2016-06-261-1/+2
| | | | | | | | | | | | | | | This variable is used by ASan (and other sanitizers in the future) on s390x-linux to override a check for CVE-2016-2143 in the running kernel (see revision 267747 on compiler-rt). Since the check simply checks if the kernel version is in a whitelist of known-good versions, it may miss distribution kernels, or manually-patched kernels - hence the need for this variable. To enable running the ASan testsuite on such kernels, this variable should be passed from the environment down to the testcases. Differential Revision: http://reviews.llvm.org/D19888 llvm-svn: 273825
* [lit] Only gather redirected files for command failures.Daniel Dunbar2016-06-101-10/+11
| | | | | | | | | | | - The intended use of this was just in diagnostics, so we shouldn't pay the cost of reading these all the time. - This will avoid including the full output of each command in tests which fail, but the most important use case for this was to gather the output of the specific command which failed. llvm-svn: 272365
* Revert "[lit] Use os.devnull instead of named temp files"Vedant Kumar2016-06-091-5/+19
| | | | | | | | | This reverts commit r272290. It breaks a test that depends on being able to seek the /dev/null equivalent on Windows: http://bb.pgr.jp/builders/ninja-clang-x64-mingw64-RA/builds/11360 llvm-svn: 272293
* [lit] Use os.devnull instead of named temp filesVedant Kumar2016-06-091-19/+5
| | | | | | | | | | | Use os.devnull instead of tempfiles when substituting '/dev/null' on Windows machines. This should make the bots just a bit speedier. Thanks to Yunzhong Gao for testing this patch on Windows! Differential Revision: http://reviews.llvm.org/D20549 llvm-svn: 272290
* [lit] Ensure we get bytes when reading redirected output files.Daniel Dunbar2016-06-081-1/+1
| | | | llvm-svn: 272147
* [lit] Ignore errors when decoding redirected output.Daniel Dunbar2016-06-071-1/+1
| | | | llvm-svn: 272066
* [lit] Fix an uninitialized var on Windows.Daniel Dunbar2016-06-071-1/+2
| | | | llvm-svn: 272052
* [utils/lit] Show available_features with --show-suites.Daniel Dunbar2016-06-071-0/+3
| | | | llvm-svn: 272022
* [lit] Improve logging with file redirection.Daniel Dunbar2016-06-074-9/+39
| | | | | | | | | - This will cause lit to automatically include the first 1K of data in redirected output files when a command fails (previously if the command failed, but the main point of the test was, say, a `FileCheck` later on, then the log wasn't helpful in showing why the command failed). llvm-svn: 272021
* [lit] Improve readability of failing scripts.Daniel Dunbar2016-06-026-14/+69
| | | | | | | | | | | | | | | | | | | | | | | - This only applies to scripts executed by the _internal_ shell script interpreter. - This patch reworks the log to look more like a shell transcript, and be less verbose (but in the interest of calling attention to the important parts). Here is an example of the new format, for commands with/without failures and with/without output: ``` $ true $ echo hi hi $ false note: command had no output on stdout or stderr error: command failed with exit status 1 ``` llvm-svn: 271610
OpenPOWER on IntegriCloud