summaryrefslogtreecommitdiffstats
path: root/llvm/utils/lit
Commit message (Collapse)AuthorAgeFilesLines
...
* Reland r374389: [lit] Clean up internal diff's encoding handlingJoel E. Denny2019-10-127-33/+86
| | | | | | | | | | To avoid breaking some tests, D66574, D68664, D67643, and D68668 landed together. However, D68664 introduced an issue now addressed by D68839, with which these are now all relanding. Differential Revision: https://reviews.llvm.org/D68664 llvm-svn: 374649
* Reland r374388: [lit] Make internal diff work in pipelinesJoel E. Denny2019-10-125-241/+276
| | | | | | | | | | To avoid breaking some tests, D66574, D68664, D67643, and D68668 landed together. However, D68664 introduced an issue now addressed by D68839, with which these are now all relanding. Differential Revision: https://reviews.llvm.org/D66574 llvm-svn: 374648
* [lit] Remove setting of the target-windows featureMartin Storsjo2019-10-121-2/+0
| | | | | | | | | No other OSes use a target-<os> feature, and no tests depend on it any lomger. Differential Revision: https://reviews.llvm.org/D68450 llvm-svn: 374639
* [lit] Small cleanups in main.pyJulian Lettner2019-10-111-25/+25
| | | | | | | | | | | | * Extract separate function for running tests from main * Push single-usage imports to point of usage * Remove unnecessary sys.exit(0) calls Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D68836 llvm-svn: 374602
* [lit] Change regex filter to ignore caseJulian Lettner2019-10-113-22/+17
| | | | | | | | | | | Make regex filter `--filter=REGEX` option more lenient via `re.IGNORECASE`. Reviewed By: yln Differential Revision: https://reviews.llvm.org/D68834 llvm-svn: 374601
* [lit] Break main into smaller functionsJulian Lettner2019-10-101-181/+199
| | | | | | | | | | | This change is purely mechanical. I will do further cleanups of parameter usages. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D68830 llvm-svn: 374452
* [lit] Add comment explaining the LIT_OPTS env var overrides command line optionsJulian Lettner2019-10-101-2/+4
| | | | | | | Normally, command line options override environment variables. Add comment to state that we are doing the reverse on purpose. llvm-svn: 374441
* [lit] Bring back `--threads` option aliasJulian Lettner2019-10-101-1/+1
| | | | | | | | Bring back `--threads` option which was lost in the move of the command line argument parsing code to cl_arguments.py. Update docs since `--workers` is preferred. llvm-svn: 374432
* Revert r374388: "[lit] Make internal diff work in pipelines"Joel E. Denny2019-10-105-276/+241
| | | | | | This breaks a Windows bot. llvm-svn: 374429
* Revert r374389: "[lit] Clean up internal diff's encoding handling"Joel E. Denny2019-10-107-86/+33
| | | | | | This breaks a Windows bot. llvm-svn: 374427
* Revert r374390: "[lit] Extend internal diff to support `-` argument"Joel E. Denny2019-10-107-146/+5
| | | | | | This breaks a Windows bot. llvm-svn: 374426
* Revert r374392: "[lit] Extend internal diff to support -U"Joel E. Denny2019-10-104-136/+6
| | | | | | This breaks a Windows bot. llvm-svn: 374425
* [lit] Leverage argparse features to remove some codeJulian Lettner2019-10-103-34/+28
| | | | | | | | Reviewed By: rnk, serge-sans-paille Differential Revision: https://reviews.llvm.org/D68589 llvm-svn: 374405
* [lit] Move argument parsing/validation to separate fileJulian Lettner2019-10-102-159/+228
| | | | | | | | Reviewed By: serge-sans-paille Differential Revision: https://reviews.llvm.org/D68529 llvm-svn: 374400
* [lit] Extend internal diff to support -UJoel E. Denny2019-10-104-6/+136
| | | | | | | | | | | | | | | | | | | | | | When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff -U1 file - ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` doesn't recognize `-U` as a command-line option. This patch adds `-U` support. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D68668 llvm-svn: 374392
* [lit] Extend internal diff to support `-` argumentJoel E. Denny2019-10-107-5/+146
| | | | | | | | | | | | | | | | | | | | | | When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff file - ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` doesn't recognize `-` as a command-line option. This patch adds support for `-` to mean stdin. Reviewed By: probinson, rnk Differential Revision: https://reviews.llvm.org/D67643 llvm-svn: 374390
* [lit] Clean up internal diff's encoding handlingJoel E. Denny2019-10-107-33/+86
| | | | | | | | | | | | | | | | | As suggested by rnk at D67643#1673043, instead of reading files multiple times until an appropriate encoding is found, read them once as binary, and then try to decode what was read. For python >= 3.5, don't fail when attempting to decode the `diff_bytes` output in order to print it. Finally, add some tests for encoding handling. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D68664 llvm-svn: 374389
* [lit] Make internal diff work in pipelinesJoel E. Denny2019-10-105-241/+276
| | | | | | | | | | | | | | | | | | | | | | | | | | | When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff file - # RUN: not diff file1 file2 | FileCheck %s ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` cannot currently be used in pipelines and doesn't recognize `-` as a command-line option. To enable pipelines, this patch moves lit's `diff` implementation into an out-of-process script, similar to lit's `cat` implementation. A follow-up patch will implement `-` to mean stdin. Reviewed By: probinson, stella.stamenova Differential Revision: https://reviews.llvm.org/D66574 llvm-svn: 374388
* [lit] Refactor ProgressDisplayJulian Lettner2019-10-095-119/+124
| | | | | | | | | | | | Move progress display to separate file. Simplify some code paths. Decouple from other components via progress callback. Remove unused `_Display` class. Reviewed By: serge-sans-paille Differential Revision: https://reviews.llvm.org/D68525 llvm-svn: 374194
* [LitConfig] Silenced notes/warnings on quiet.Andrew Trick2019-10-081-2/+4
| | | | | | | | | | | | | | | | | | | | Lit has a "quiet" option, -q, which is documented to "suppress no error output". Previously, LitConfig displayed notes and warnings when the quiet option was specified. The result was that it was not possible to get only pertinent file/line information to be used by an editor to jump to the location where checks were failing without passing a number of unhelpful locations first. Here, the implementations of LitConfig.note and LitConfig.warning are modified to account for the quiet flag and avoid displaying if the flag has indeed been set. Patch by Nate Chandler Reviewed by yln Differential Revision: https://reviews.llvm.org/D68044 llvm-svn: 374009
* [lit] Use better name for "test in parallel" conceptJulian Lettner2019-10-044-25/+22
| | | | | | | | | | | | In the past, lit used threads to run tests in parallel. Today we use `multiprocessing.Pool`, which uses processes. Let's stay more abstract and use "worker" everywhere. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D68475 llvm-svn: 373794
* Revert "[lit] Add -D__clang_analyzer__ to clang_analyze_cc1"Jan Korous2019-09-241-1/+1
| | | | | | This reverts commit 4185460f758b98ea5b898c04c179704756ca8f53. llvm-svn: 372686
* [lit] Add -D__clang_analyzer__ to clang_analyze_cc1Jan Korous2019-09-241-1/+1
| | | | | | Fixup after fbd13570b0d llvm-svn: 372682
* Revert r372035: "[lit] Make internal diff work in pipelines"Joel E. Denny2019-09-165-276/+241
| | | | | | This breaks a Windows bot. llvm-svn: 372051
* [lit] Make internal diff work in pipelinesJoel E. Denny2019-09-165-241/+276
| | | | | | | | | | | | | | | | | | | | | | | | | | | When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff file - # RUN: not diff file1 file2 | FileCheck %s ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` cannot currently be used in pipelines and doesn't recognize `-` as a command-line option. To enable pipelines, this patch moves lit's `diff` implementation into an out-of-process script, similar to lit's `cat` implementation. A follow-up patch will implement `-` to mean stdin. Reviewed By: probinson, stella.stamenova Differential Revision: https://reviews.llvm.org/D66574 llvm-svn: 372035
* [lit] Diagnose insufficient args to internal envJoel E. Denny2019-08-226-3/+31
| | | | | | | | | | | | | Without this patch, failing to provide a subcommand to lit's internal `env` results in either a python `IndexError` or an attempt to execute the final `env` argument, such as `FOO=1`, as a command. This patch diagnoses those cases with a more helpful message. Reviewed By: stella.stamenova Differential Revision: https://reviews.llvm.org/D66482 llvm-svn: 369620
* [lit] Check for accidental external command callsJoel E. Denny2019-08-198-0/+48
| | | | | | | | | | | | | | | This patch extends lit's test suite to check that lit's internal shell doesn't accidentally execute internal commands as external commands. It does so by putting fake failing versions of those commands in `PATH` while the entire lit test suite is running. Without the fixes in D65697 but with its tests, this approach catches accidental external `env` calls. Reviewed By: probinson Differential Revision: https://reviews.llvm.org/D66293 llvm-svn: 369309
* lit: Bump version to 0.10.0Tom Stellard2019-08-081-1/+1
| | | | | | | | | | | | Reviewers: hans Subscribers: hans, delcypher, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D65763 llvm-svn: 368329
* lit: Use a License classifier that pypi will acceptTom Stellard2019-08-081-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | Summary: 'OSI Approved :: Apache-2.0 with LLVM exception' is not a valid classifier. 'OSI Approved :: Apache Software License' is the closest fit for the new license, so we've decided to use this one. The classifiers seem to only be used for searching on the pypi website, so this does not actually change the license of the code. We still pass 'Apache-2.0 with LLVM exception' as the license to setup(), and this appears alongside the classifier on the pypi webpage for lit. Reviewers: chandlerc, ddunbar, joerg Reviewed By: joerg Subscribers: delcypher, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D65762 llvm-svn: 368315
* [lit] Print internal env commandsJoel E. Denny2019-08-032-9/+30
| | | | | | | | | | | | Without this patch, the internal `env` command removes `env` and its args from the command line while parsing it. This patch modifies a copy instead so that the original command line is printed. Reviewed By: stella.stamenova, rnk Differential Revision: https://reviews.llvm.org/D65624 llvm-svn: 367752
* [lit] Actually run tests for internal env commandJoel E. Denny2019-08-032-1/+1
| | | | | | | | | | | Put the main test script in the right directory, and fix a python bug in a local script. Reviewed By: stella.stamenova Differential Revision: https://reviews.llvm.org/D65623 llvm-svn: 367751
* [lit] Fix 42812: lit test suite can no longer be run stand-aloneStella Stamenova2019-08-021-1/+12
| | | | | | | | | | | | | | | | | | | Summary: This change updates the lit.cfg file to use llvm_config when it is available, but when it is not, it directly modifies the config object. This makes it possible to run the lit tests standalone without having built llvm (as long as the correct binaries are present in the path such as FileCheck and not). Because the lit tests don't take a hard dependency on llvm_config, some features such as system-windows have to have definitions in lit's cfg file as well. This is a potential issue as the os features sometimes change names (for example, we went from windows to system-windows, etc.). This can cause drift between lit's tests and the rest of the llvm tests. Reviewers: probinson, mgorny Reviewed By: mgorny Subscribers: delcypher, llvm-commits, asmith Tags: #llvm Differential Revision: https://reviews.llvm.org/D65674 llvm-svn: 367730
* Revert r366980: "[lit] Protect full test suite from FILECHECK_OPTS"Joel E. Denny2019-08-012-10/+3
| | | | | | Windows bots are broken. See recent D65335 and D65156 comments. llvm-svn: 367627
* Revert r367123: "[llvm] [lit/tests] Replace 'env -u' with more portable ↵Joel E. Denny2019-08-011-1/+1
| | | | | | | | | construct" Must be reverted in order to revert r366980, which breaks windows bots. See recent D65335 and D65156 comments. llvm-svn: 367626
* [llvm] [lit/tests] Replace 'env -u' with more portable constructMichal Gorny2019-07-261-1/+1
| | | | | | | | | | | Set environment variables to empty values rather than attempting to unset them via 'env -u', in order to fix NetBSD test regression caused by r366980. POSIX does not guarantee that env(1) supports '-u' option, and indeed NetBSD env(1) does not support it. Differential Revision: https://reviews.llvm.org/D65335 llvm-svn: 367123
* [lit] Protect full test suite from FILECHECK_OPTSJoel E. Denny2019-07-252-3/+10
| | | | | | | | | | | | | | | | | | | | | lit's test suite calls lit multiple times for various sample test suites. `FILECHECK_OPTS` is safe for FileCheck calls in lit's test suite. It's not safe for FileCheck calls in the sample test suites, whose output affects the results of lit's test suite. Without this patch, only one such sample test suite is protected from `FILECHECK_OPTS`, and I admit I haven't discovered other cases for which I can produce false failures using `FILECHECK_OPTS`. However, it's hard to predict the future, especially false passes. Thus, this patch protects all existing and future sample test suites from `FILECHECK_OPTS` (and the deprecated `FILECHECK_DUMP_INPUT_ON_FAILURE`). Reviewed By: probinson Differential Revision: https://reviews.llvm.org/D65156 llvm-svn: 366980
* [AIX][lit] Don't depend on psutil on AIXDavid Tenty2019-07-245-37/+71
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: On AIX psutil can run into problems with permissions to read the process tree, which causes problems for python timeout tests which need to kill off a test and it's children. This patch adds a workaround by invoking shell via subprocess and using a platform specific option to ps to list all the descendant processes so we can kill them. We add some checks so lit can tell whether timeout tests are supported with out exposing whether we are utilizing the psutil implementation or the alternative. Reviewers: hubert.reinterpretcast, andusy, davide, delcypher Reviewed By: delcypher Subscribers: davide, delcypher, christof, lldb-commits, libcxx-commits, llvm-commits Tags: #lldb, #libc, #llvm Differential Revision: https://reviews.llvm.org/D64251 llvm-svn: 366912
* [LIT] Emit timeout error message only if timeout was reachedAlexey Bader2019-07-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | Summary: This improves readability of LIT output: previously error messages gets emitted that say that there was no error: error: command reached timeout: False Patch by Alexey Sachkov. Reviewers: ddunbar, mgorny, modocache Reviewed By: mgorny Subscribers: delcypher, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D64240 llvm-svn: 365895
* NFC: lit: python3 compatibility for functionsAlex Lorenz2019-07-121-2/+2
| | | | | | | | usePlatformSdkOnDarwin/findPlatformSdkVersionOnMacOS These functions should decode subprocess output llvm-svn: 365852
* [lit] Parse command-line options from LIT_OPTSJoel E. Denny2019-07-084-1/+44
| | | | | | | | | | | | | | | | | Similar to `FILECHECK_OPTS` for FileCheck, `LIT_OPTS` makes it easy to adjust lit behavior when running the test suite via ninja. For example: ``` $ LIT_OPTS='--time-tests -vv --filter=threadprivate' \ ninja check-clang-openmp ``` Reviewed By: probinson Differential Revision: https://reviews.llvm.org/D64135 llvm-svn: 365313
* Fix some lit test ResourceWarnings on WindowsAdrian McCarthy2019-06-181-0/+1
| | | | | | | | | | | | | | | | | | When running LLDB lit tests on Windows, the system selects a debug version of Python, which was issuing lots of ResourceWarnings about files that weren't closed. There are two kinds of them, and each test triggered one of each. This patch fixes one kind by ensuring TestRunner explicitly close the temporary files created for routing stderr. This is important on Windows but has no net effect on Posix systems. The remaining ResourceWarnings are more elusive; the bug may lie in the Python library subprocess.py, and it may be Windows-specific. Differential Revision: https://reviews.llvm.org/D63102 llvm-svn: 363700
* [lit] Fix UnicodeEncodeError when test commands contain non-ASCII charsMichal Gorny2019-06-143-4/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | Ensure that the bash script written by lit TestRunner is open with UTF-8 encoding when using Python 3. Otherwise, attempt to write non-ASCII characters causes UnicodeEncodeError. This happened e.g. with the following LLD test: UNRESOLVED: lld :: ELF/format-binary-non-ascii.s (657 of 2119) ******************** TEST 'lld :: ELF/format-binary-non-ascii.s' FAILED ******************** Exception during script execution: Traceback (most recent call last): File "/home/mgorny/llvm-project/llvm/utils/lit/lit/worker.py", line 63, in _execute_test result = test.config.test_format.execute(test, lit_config) File "/home/mgorny/llvm-project/llvm/utils/lit/lit/formats/shtest.py", line 25, in execute self.execute_external) File "/home/mgorny/llvm-project/llvm/utils/lit/lit/TestRunner.py", line 1644, in executeShTest res = _runShTest(test, litConfig, useExternalSh, script, tmpBase) File "/home/mgorny/llvm-project/llvm/utils/lit/lit/TestRunner.py", line 1590, in _runShTest res = executeScript(test, litConfig, tmpBase, script, execdir) File "/home/mgorny/llvm-project/llvm/utils/lit/lit/TestRunner.py", line 1157, in executeScript f.write('{ ' + '; } &&\n{ '.join(commands) + '; }') UnicodeEncodeError: 'ascii' codec can't encode character '\xa3' in position 274: ordinal not in range(128) Differential Revision: https://reviews.llvm.org/D63254 llvm-svn: 363388
* Revert r363298 "[lit] Disable test on darwin when building shared libs."Nico Weber2019-06-141-2/+0
| | | | | | It broke running check-lld on mac, see https://reviews.llvm.org/D61697 llvm-svn: 363379
* [lit] Disable test on darwin when building shared libs.Don Hinton2019-06-131-0/+2
| | | | | | | | | | | | | | | | | | | | | Summary: This test fails to link shared libraries because tries to run a copied version of clang-check to see if the mock version of libcxx in the same directory can be loaded dynamically. Since the test is specifically designed not to look in the default just-built lib directory, it must be disabled when building with BUILD_SHARED_LIBS=ON. Currently only disabling it on Darwin and basing it on the enable_shared flag. Reviewed By: ilya-biryukov Tags: #clang, #llvm Differential Revision: https://reviews.llvm.org/D61697 llvm-svn: 363298
* Fix string literals to avoid deprecation warnings in regexp patternsAdrian McCarthy2019-06-071-3/+3
| | | | | | | | | | | | | In LLDB, where tests run with the debug version of Python, we get a series of deprecation warnings because escape sequences like `\(` are being treated as part of the string literal rather than an escape for the regexp pattern. NFC intended. Differential Revision: https://reviews.llvm.org/D62882 llvm-svn: 362846
* lit: modernize the lit configuration for the lit testsStella Stamenova2019-05-293-33/+17
| | | | | | | | | | | | | | Summary: This also normalizes the config feature that represents the windows platform to "system-windows" as opposed to having both "windows" and "system-windows" Reviewers: asmith, probinson Subscribers: delcypher, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D61798 llvm-svn: 361998
* Stop defining negative versions of some lit feature keywords:Paul Robinson2019-05-131-9/+8
| | | | | | | | | | | | | | | | | | | zlib/nozlib, asan/not_asan, msan/not_msan, ubsan/not_ubsan. We still have two other ways to express the absence of a feature. First, we have the '!' operator to invert the sense of a keyword. For example, given a feature that depends on zlib being unavailable, its test can say: REQUIRES: !zlib Second, if a test doesn't play well with some features, such as sanitizers, that test can say: UNSUPPORTED: asan, msan The different ways of writing these exclusions both have the same technical effect, but have different implications to the reader. llvm-svn: 360603
* Use UNSUPPORTED: windows in shtest-timeout.py. Apparently system-windows ↵Stella Stamenova2019-05-091-1/+3
| | | | | | does not cover all cases either and the case it doesn't cover affects one of the buildbots. llvm-svn: 360373
* Use UNSUPPORTED: system-windows instead of REQUIRES: nowindows or ↵Stella Stamenova2019-05-092-2/+2
| | | | | | UNSUPPORTED: windows. nowindows is not currently defined and windows does not cover all cases. system-windows is also consistent with how other platforms are used. llvm-svn: 360368
* Re-enable lit test shtest-timeout.py on non-Windows.Paul Robinson2019-05-091-1/+1
| | | | | | It was disabled incorrectly, which meant it wasn't running anywhere. llvm-svn: 360356
OpenPOWER on IntegriCloud