summaryrefslogtreecommitdiffstats
path: root/lldb/packages/Python/lldbsuite/test/dotest_args.py
Commit message (Collapse)AuthorAgeFilesLines
* dotest.py: Add option to pass extra lldb settings to dotestAdrian Prantl2020-01-141-0/+8
| | | | | | | The primary motivation for this is to add another dimension to the Swift LLDB test matrix, but this seems generally useful. Differential Revision: https://reviews.llvm.org/D72662
* [lldb][tests] Make it possible to expect failure for a whole categoryTatyana Krasnukha2020-01-101-0/+6
| | | | | | | | | | | | | There already are decorators and "--excluded" option to mark test-cases/files as expected to fail. However, when a new test file is added and it which relates to a feature that a target doesn't support, this requires either adding decorators to that file or modifying the file provided as "--excluded" option value. The purpose of this patch is to avoid any modifications in such cases. E.g. if a target doesn't support "watchpoints" and passes "--xfail-category watchpoint" to dotest, a testing job will not fail after a new watchpoint-related test file is added. Differential Revision: https://reviews.llvm.org/D71906
* [lldb][tests][NFC] Unify variable naming conventionTatyana Krasnukha2020-01-101-2/+2
|
* [lldb][NFC] Remove 'from __future__ import print_function' from all tests ↵Raphael Isemann2019-12-131-1/+0
| | | | | | | | | | | | | | | | | | | | | that don't actually call 'print()' Summary: A lot of tests do this trick but the vast majority of them don't even call `print()`. Most of this patch was generated by a script that just looks at all the files and deletes the line if there is no `print (` or `print(` anywhere else in the file. I checked the remaining tests manually and deleted the import if we never call print (but instead do stuff like `expr print(...)` and similar false-positives). I also corrected the additional empty lines after the import in the files that I manually edited. Reviewers: JDevlieghere, labath, jfb Reviewed By: labath Subscribers: dexonsmith, wuzish, nemanjai, kbarton, christof, arphaman, abidh, lldb-commits Tags: #lldb Differential Revision: https://reviews.llvm.org/D71452
* [test] Use a different module cache for Shell and API tests.Jonas Devlieghere2019-10-101-3/+8
| | | | | | | | | | | | | | Before the test reorganization, everything was part of a single test suite with a single module cache. Now that things are properly separated this is no longer the case. Only the shell tests inherited the logic to properly configure and wipe the module caches. This patch adds that logic back for the API tests. While doing so, I noticed that we were configuring a Clang module cache in CMake, but weren't actually using it from dotest.py. I included a fix for that in this patch as well. Differential revision: https://reviews.llvm.org/D68755 llvm-svn: 374386
* DWIMy filterspecs for dotest.pyLawrence D'Anna2019-10-081-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: dotest.py currently requires a filterspec to be of the form `TestCase.test_method`. This patch makes it more flexible, so you can pass `TestModule.TestCase.test_method` or `TestModule.TestCase` or `TestCase.test_method` or just `test_method`. This makes it more convenient to just copy a test name out of the terminal after running a bunch of tests and use it as a filterspec. Reviewers: JDevlieghere, jasonmolenda, labath Reviewed By: JDevlieghere Subscribers: jingham, lldb-commits Tags: #lldb Differential Revision: https://reviews.llvm.org/D68545 llvm-svn: 373997
* [test] Fix various module cache bugs and inconsistenciesJonas Devlieghere2019-08-291-0/+5
| | | | | | | | | | | | | | | | | Currently, lit tests don't set neither the module cache for building inferiors nor the module cache used by lldb when running tests. Furthermore, we have several places where we rely on the path to the module cache being always the same, rather than passing the correct value around. This makes it hard to specify a different module cache path when debugging a a test. This patch reworks how we determine and pass around the module cache paths and fixes the omission on the lit side. It also adds a sanity check to the lit and dotest suites. Differential revision: https://reviews.llvm.org/D66966 llvm-svn: 370394
* [dotest] Remove the curses result formatter.Jonas Devlieghere2019-08-291-4/+0
| | | | | | | | | | This removes the curses result formatter which appears to be broken. Passing --curses to dotest.py screws up my terminal and doesn't run any tests. It even crashes Python on occasion. Differential revision: https://reviews.llvm.org/D66917 llvm-svn: 370386
* [dotest] Make dotest.py invocation repeatableJonas Devlieghere2019-08-281-22/+0
| | | | | | | | | | | This removes support for reading the LLDB_TEST_ARGUMENTS environment variable and instead requires all arguments to be specified as part of the invocation. This ensures that dotest.py invocations are easily repeatable. Differential revision: https://reviews.llvm.org/D66912 llvm-svn: 370278
* [dotest] Remove --event-add-entriesJonas Devlieghere2019-08-281-7/+0
| | | | | | | | This argument was used by dosep.py to pass information around from the workers. With dosep.py gone, I'm fairly sure we don't need this any longer. llvm-svn: 370266
* [dotest] Remove --rerun-max-file-thresholdJonas Devlieghere2019-08-281-9/+0
| | | | | | This variable corresponding to this argument is set but never read. llvm-svn: 370264
* [dotest] Remove -q (quiet) flag.Jonas Devlieghere2019-08-281-1/+0
| | | | | | | | | | | | | | This patch removes the -q (quiet) flag and changing the default behavior. Currently the flag serves two purposes that are somewhat contradictory, as illustrated by the difference between the argument name (quiet) and the configuration flag (parsable). On the one hand it reduces output, but on the other hand it prints more output, like the result of individual tests. My proposal is to guard the extra output behind the verbose flag and always print the individual test results. Differential revision: https://reviews.llvm.org/D66837 llvm-svn: 370226
* [dotest] Remove results portJonas Devlieghere2019-08-271-6/+0
| | | | | | | | | | The results port was used by dosep.py to deal with test results coming form different processes. With dosep.py gone, I don't think we need this any longer. Differential revision: https://reviews.llvm.org/D66811 llvm-svn: 370090
* [dotest] Remove long running test "decorator" and re-enable tests.Jonas Devlieghere2019-08-271-1/+0
| | | | | | | | | | | | Today I discovered the skipLongRunningTest decorator and to my surprise all the tests were passing without the decorator. They don't seem to be that expensive either, they take a few seconds but we have tests that take much longer than that. As such I propose to remove the decorator and enable them by default. Differential revision: https://reviews.llvm.org/D66774 llvm-svn: 369995
* [dotest] Add --dwarf-version to override the tested DWARF version.Jonas Devlieghere2019-08-191-0/+6
| | | | | | | | | | | | | On the matrix bot on GreenDragon [1] we want to run the test suite against different DWARF versions. The idea here is not to replace targeted tests for certain DWARF features, but rather to provide an easy way to support this configuration. [1] http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake-matrix/ Differential revision: https://reviews.llvm.org/D66370 llvm-svn: 369272
* [dotest] Remove multiprocessingJonas Devlieghere2019-07-301-39/+0
| | | | | | | | | | | Now that the Xcode project is removed, I want to focus on dotest as a test framework, and remove its driver capabilities for which we already rely on llvm's lit. Removing multiprocessing is the first step in that direction. Differential revision: https://reviews.llvm.org/D65311 llvm-svn: 367331
* [dotest] Add the ability to set environment variables for the inferior.Jonas Devlieghere2019-06-261-0/+6
| | | | | | | | | | | This patch adds a dotest flag for setting environment variables for the inferior. This is different from the current --env flag, which sets variables in the debugger's environment. This allows us to set things like LD_LIBRARY_PATH for testing. Differential revision: https://reviews.llvm.org/D63790 llvm-svn: 364443
* Allow use of self.filecheck in LLDB tests (c.f self.expect)Vedant Kumar2018-09-181-0/+2
| | | | | | | | | | | | | Add a "filecheck" method to the LLDB test base. This allows test authors to pattern match command output using FileCheck, making it possible to write stricter tests than what `self.expect` allows. For context (motivation, examples of stricter checking, etc), see the lldb-dev thread: "Using FileCheck in lldb inline tests". Differential Revision: https://reviews.llvm.org/D50751 llvm-svn: 342508
* [dotest] Fix syntax error and typo.Jonas Devlieghere2018-04-121-1/+1
| | | | | | Python uses `elif` rather than `else if`. Fixes r329889. llvm-svn: 329890
* [dotest] Use in-tree dsymutil on DarwinJonas Devlieghere2018-04-121-0/+2
| | | | | | | | | | | | | | | | | | | Summary: With the upstream implementation of dsymutil containing almost all functionality from the one shipped with Xcode, we want to use the in-tree version for running the test suite. This will also allow us to re-enable TestUnicodeSymbols which was failing because of the discrepancy in how Unicode symbols were hashed in lldb and older versions of dsymutil. Reviewers: aprantl, davide, jingham, labath Subscribers: mgorny, llvm-commits, lldb-commits Differential Revision: https://reviews.llvm.org/D45518 llvm-svn: 329889
* [test] Skip a test when using an out-of-tree debugserverVedant Kumar2018-03-081-0/+5
| | | | | | | | | | | | | | | | | | | | The test "test_fp_special_purpose_register_read" in TestRegisters.py fails on Darwin machines configured to use an out-of-tree debugserver. The error message is: 'register read ftag' returns expected result, got 'ftag = 0x80'. This indicates that the debugserver in use is too old. This commit introduces a decorator which can be used to skip tests which rely on having a just-built debugserver. This resolves the issue: $ ./bin/llvm-dotest -p TestRegisters.py -v 1 out of 617 test suites processed - TestRegisters.py Test Methods: 7 Success: 6 Skip: 1 ... llvm-svn: 327052
* Use an alternative approach to prevent Spotlight from indexing the build ↵Adrian Prantl2018-02-021-1/+1
| | | | | | directory. llvm-svn: 324115
* Compile the LLDB tests out-of-tree.Adrian Prantl2018-01-301-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | This patch is the result of a discussion on lldb-dev, see http://lists.llvm.org/pipermail/lldb-dev/2018-January/013111.html for background. For each test (should be eventually: each test configuration) a separate build directory is created and we execute make VPATH=$srcdir/path/to/test -C $builddir/path/to/test -f $srcdir/path/to/test/Makefile -I $srcdir/path/to/test In order to make this work all LLDB tests need to be updated to find the executable in the test build directory, since CWD still points at the test's source directory, which is a requirement for unittest2. Although we have done extensive testing, I'm expecting that this first attempt will break a few bots. Please DO NOT HESITATE TO REVERT this patch in order to get the bots green again. We will likely have to iterate on this some more. Differential Revision: https://reviews.llvm.org/D42281 llvm-svn: 323803
* dotest.py: remove the ability to specify different architectures/compilers ↵Pavel Labath2017-03-151-3/+2
| | | | | | | | | | | | | | | | | | | | | | | | | in a single invocation Summary: This has been broken at least since the new test result framework was added, which was over a year ago. It looks like nobody has missed it since. Removing this makes the gmodules handling code saner, as it already did not know how to handle the multiple-compilers case. My motivation for this is libc++ data formatters support on android -- I am trying make a central way of determining whether libc++ tests can be run, and without this, I would have to resort to similar hacks as the gmodules code. Reviewers: jingham, zturner Subscribers: danalbert, tfiala, lldb-commits Differential Revision: https://reviews.llvm.org/D30779 llvm-svn: 297811
* [CMake] Override debugserver to use the build tree on DarwinChris Bieneman2017-03-141-0/+4
| | | | | | This patch adds support to the test suite for overriding the path to debugserver, and uses the override to point to the build tree's debugserver on Darwin. llvm-svn: 297776
* [Test Suite] Allow overriding codesign identityChris Bieneman2016-10-211-0/+5
| | | | | | | | | | | | Summary: Not everyone names their code sign identity "lldb_codesign", so it is nice to allow this to be overridden. Reviewers: zturner, tfiala Subscribers: labath, mgorny, lldb-commits Differential Revision: https://reviews.llvm.org/D25714 llvm-svn: 284893
* Fix building tests without system headers on DarwinChris Bieneman2016-10-121-1/+1
| | | | | | | | | | | | Summary: Default installations of OS X do not have system headers installed at /usr/include. This patch allows the LLDB test executables to properly compile when built on a system without headers at /usr/include by specifying a default value for the apple-sdk flag as "macosx". Reviewers: tfiala, zturner Subscribers: lldb-commits Differential Revision: https://reviews.llvm.org/D25487 llvm-svn: 284042
* Improvements to testing blacklistFrancis Ricci2016-10-041-1/+1
| | | | | | | | | | | | | | | | | | | Summary: This patch is necessary because individual test cases are not required to have unique names. Therefore, test cases must now be specified explicitly in the form <TestCase>.<TestMethod>. Because it works by regex matching, passing just <TestCase> will still disable an entire file. This also allows for multiple exclusion files to be specified. Reviewers: zturner, labath, jingham, tfiala Subscribers: lldb-commits, sas Differential Revision: https://reviews.llvm.org/D24988 llvm-svn: 283238
* Allow for tests to be disabled at runtimeFrancis Ricci2016-09-231-0/+3
| | | | | | | | | | | | | | | | Summary: The current implementation of the test suite allows the user to run a certain subset of tests using '-p', but does not allow the inverse, where a user wants to run all but some number of known failing tests. Implement this functionality. Reviewers: labath, zturner, tfiala Subscribers: jingham, sas, lldb-commits Differential Revision: https://reviews.llvm.org/D24629 llvm-svn: 282298
* *** This commit represents a complete reformatting of the LLDB source codeKate Stone2016-09-061-29/+131
| | | | | | | | | | | | | | | | | | | | | | | *** to conform to clang-format’s LLVM style. This kind of mass change has *** two obvious implications: Firstly, merging this particular commit into a downstream fork may be a huge effort. Alternatively, it may be worth merging all changes up to this commit, performing the same reformatting operation locally, and then discarding the merge for this particular commit. The commands used to accomplish this reformatting were as follows (with current working directory as the root of the repository): find . \( -iname "*.c" -or -iname "*.cpp" -or -iname "*.h" -or -iname "*.mm" \) -exec clang-format -i {} + find . -iname "*.py" -exec autopep8 --in-place --aggressive --aggressive {} + ; The version of clang-format used was 3.9.0, and autopep8 was 1.2.4. Secondly, “blame” style tools will generally point to this commit instead of a meaningful prior commit. There are alternatives available that will attempt to look through this change and find the appropriate prior commit. YMMV. llvm-svn: 280751
* Allow custom formatting of session log file names.Zachary Turner2016-05-171-0/+2
| | | | | | Differential Revision: http://reviews.llvm.org/D20306 llvm-svn: 269793
* test infra: enable single-worker rerun phase for flakey tests.Todd Fiala2015-12-141-0/+11
| | | | | | | | | | | | | | | | Use of --rerun-all-issues will enable any test method failure, not just test methods marked with the flakey decorator, to rerun. Currently this does not change the flakey logic's immediate rerun attempt. I want to make sure this doesn't cause any significant issues before changing that part. The rerun reporting is only known to work properly with the default (new) BasicResultsFormatter reporting. Once we work out any issues, I'll go back and make sure the curses output handles it properly as well. llvm-svn: 255543
* Make debug info specification use categories system.Zachary Turner2015-12-141-1/+0
| | | | | | | Reviewed By: Tamas Berghammer, Pavel Labath Differential Revision: http://reviews.llvm.org/D15428 llvm-svn: 255525
* test infra: adds book-keeping for rerunnable testsTodd Fiala2015-12-121-0/+9
| | | | | | | | | | | | | | Also adds full path info for exceptional exits and timeouts when no test method is currently running. Adds --rerun-all-issues command line arg. If specified, all test issues are eligible for rerun. If not specified, only tests marked flakey are eligible for rerun. The actual rerunning will occur in an upcoming change. This change just handles tha accounting of what should be rerun. llvm-svn: 255438
* Remove -S option from dotest.py.Zachary Turner2015-12-111-1/+0
| | | | llvm-svn: 255361
* Remove -r and -R options from dotest.py.Zachary Turner2015-12-111-2/+0
| | | | llvm-svn: 255360
* Remove the -x option from dotest.py.Zachary Turner2015-12-101-1/+0
| | | | llvm-svn: 255279
* Remove deprecated command line options from dotest.pyZachary Turner2015-12-101-9/+0
| | | | llvm-svn: 255278
* Remove the --output-on-success command line argument from dotest.Zachary Turner2015-12-101-7/+0
| | | | llvm-svn: 255277
* Remove the -T option from dotest.py.Zachary Turner2015-12-101-1/+0
| | | | llvm-svn: 255276
* Remove -w option from dotest.py.Zachary Turner2015-12-101-1/+0
| | | | llvm-svn: 255275
* Remove the -P option from dotest.pyZachary Turner2015-12-091-1/+0
| | | | | | | | This was an option to display a graphical progress bar. Nobody is using this, and it doesn't work correctly anyway with the new result formatter. llvm-svn: 255153
* Delete the -n command line option from dotest.py.Zachary Turner2015-12-091-1/+0
| | | | | | | This removes the option to not print some one time version and diagnostic information when running the test suite. llvm-svn: 255152
* Remove the -i command line option from dotest.py.Zachary Turner2015-12-091-1/+0
| | | | | | | This is part of a larger effort to remove unused command line options from dotest.py. llvm-svn: 255151
* Delete the -F command line option from dotest.py.Zachary Turner2015-12-091-1/+0
| | | | | | | | This removes the failfast command line option as part of an effort to simplify dotest and remove unused command line options. You can still Ctrl+C any time you want to exit early. llvm-svn: 255150
* Remove -k command line option from dotest.py.Zachary Turner2015-12-091-1/+0
| | | | | | This is part of an effort to remove unused command line options. llvm-svn: 255143
* Remove -e option from dotest.py.Zachary Turner2015-12-091-1/+0
| | | | | | | This is part of an effort to clean up dotest command line options that are no longer used. llvm-svn: 255142
* Remove -libcxx option from dotest.pyZachary Turner2015-12-091-1/+0
| | | | | | | Nobody was using this, and plus it can be achieved just as well by using -E to set an environment variable. llvm-svn: 255141
* Remove the -c option from dotest.py.Zachary Turner2015-12-081-1/+0
| | | | | | | | | | This seems to be a legacy relic from days gone by where the remote test suite runner operated completely differently than it does today. git blames and comments traced this functionality back to about 2012, and nobody seems to know anything about it now. llvm-svn: 255060
* Remove the -X option from dotest.pyZachary Turner2015-12-081-1/+0
| | | | | | | | This removes the option to exclude a single directory. This is part of an effort to remove unused options and cleanup the interface to the test suite. llvm-svn: 255048
OpenPOWER on IntegriCloud