summaryrefslogtreecommitdiffstats
path: root/lldb/packages/Python/lldbsuite/test/dotest.py
Commit message (Collapse)AuthorAgeFilesLines
* [dotest] Finish removing -qPavel Labath2019-08-301-1/+1
| | | | | | | One usage of this option remained, and caused dotest to error out if one happened to pass the -v flag. llvm-svn: 370462
* [test] Fix various module cache bugs and inconsistenciesJonas Devlieghere2019-08-291-14/+19
| | | | | | | | | | | | | | | | | Currently, lit tests don't set neither the module cache for building inferiors nor the module cache used by lldb when running tests. Furthermore, we have several places where we rely on the path to the module cache being always the same, rather than passing the correct value around. This makes it hard to specify a different module cache path when debugging a a test. This patch reworks how we determine and pass around the module cache paths and fixes the omission on the lit side. It also adds a sanity check to the lit and dotest suites. Differential revision: https://reviews.llvm.org/D66966 llvm-svn: 370394
* [dotest] Remove deprecated loggin through env variables.Jonas Devlieghere2019-08-291-72/+0
| | | | | | | | | | It used to be possible to enable logging through environment variables read by dotest. This approach is deprecated, as stated in the dotest help output. Instead --channel should be used. Differential revision: https://reviews.llvm.org/D66920 llvm-svn: 370387
* [dotest] Remove the curses result formatter.Jonas Devlieghere2019-08-291-6/+0
| | | | | | | | | | This removes the curses result formatter which appears to be broken. Passing --curses to dotest.py screws up my terminal and doesn't run any tests. It even crashes Python on occasion. Differential revision: https://reviews.llvm.org/D66917 llvm-svn: 370386
* [dotest] Make dotest.py invocation repeatableJonas Devlieghere2019-08-281-7/+7
| | | | | | | | | | | This removes support for reading the LLDB_TEST_ARGUMENTS environment variable and instead requires all arguments to be specified as part of the invocation. This ensures that dotest.py invocations are easily repeatable. Differential revision: https://reviews.llvm.org/D66912 llvm-svn: 370278
* [dotest] Remove --event-add-entriesJonas Devlieghere2019-08-281-18/+0
| | | | | | | | This argument was used by dosep.py to pass information around from the workers. With dosep.py gone, I'm fairly sure we don't need this any longer. llvm-svn: 370266
* [dotest] Remove --rerun-max-file-thresholdJonas Devlieghere2019-08-281-1/+0
| | | | | | This variable corresponding to this argument is set but never read. llvm-svn: 370264
* [dotest] Centralize and simplify session dir logic (NFC)Jonas Devlieghere2019-08-281-24/+11
| | | | | | | | I was looking at the session directory logic for unrelated reasons and noticed that the logic spread out across dotest. This simplifies things a bit by moving the logic together. llvm-svn: 370259
* [dotest] Remove outdates TODOJonas Devlieghere2019-08-281-1/+0
| | | | | | The referenced function `find_test_files_in_dir_tree` no longer exists. llvm-svn: 370235
* [dotest] Don't try to guess the lldb binary & python dir.Jonas Devlieghere2019-08-281-94/+17
| | | | | | | | | | Now that all supported build systems create a valid dotest.py invocation, we no longer need to guess the location of the lldb binary and Python directory. Differential revision: https://reviews.llvm.org/D66896 llvm-svn: 370234
* [dotest] Don't try to guess the llvm binary dir.Jonas Devlieghere2019-08-281-35/+0
| | | | | | | | | | | | | | | Now that all supported build systems create a valid dotest.py invocation, we no longer need to guess the directory where any of the llvm tools live. Additionally, the current logic is incomplete: it doesn't try to find any other tools than FileCheck, such as dsymutil for example. If no FileCheck is provided, we should print a warning and skip the tests that need it, but that's not part of this patch. Differential revision: https://reviews.llvm.org/D66893 llvm-svn: 370232
* [dotest] Don't spend time replacing spaces in print output.Jonas Devlieghere2019-08-281-8/+0
| | | | | | | Replacing all spaces with dashes seems like a lot of needless work for a string that's just printed. llvm-svn: 370231
* [dotest] Remove -q (quiet) flag.Jonas Devlieghere2019-08-281-15/+7
| | | | | | | | | | | | | | This patch removes the -q (quiet) flag and changing the default behavior. Currently the flag serves two purposes that are somewhat contradictory, as illustrated by the difference between the argument name (quiet) and the configuration flag (parsable). On the one hand it reduces output, but on the other hand it prints more output, like the result of individual tests. My proposal is to guard the extra output behind the verbose flag and always print the individual test results. Differential revision: https://reviews.llvm.org/D66837 llvm-svn: 370226
* [dotest] Remove check for LLDB_TESTSUITE_FORCE_FINISHJonas Devlieghere2019-08-271-6/+0
| | | | llvm-svn: 370120
* [dotest] Remove results portJonas Devlieghere2019-08-271-10/+0
| | | | | | | | | | The results port was used by dosep.py to deal with test results coming form different processes. With dosep.py gone, I don't think we need this any longer. Differential revision: https://reviews.llvm.org/D66811 llvm-svn: 370090
* [dotest] Remove long running test "decorator" and re-enable tests.Jonas Devlieghere2019-08-271-7/+0
| | | | | | | | | | | | Today I discovered the skipLongRunningTest decorator and to my surprise all the tests were passing without the decorator. They don't seem to be that expensive either, they take a few seconds but we have tests that take much longer than that. As such I propose to remove the decorator and enable them by default. Differential revision: https://reviews.llvm.org/D66774 llvm-svn: 369995
* [dotest] Print invocation when encountering an error.Jonas Devlieghere2019-08-261-2/+6
| | | | | | | | | | | With this patch dotest.py will print the full invocation whenever it fails to parse its arguments. The dotest invocation is usually build up with different inputs, potentially coming from CMake, lldb-dotest, lit or passed directly. This can make debugging hard, especially on CI, where there might be another layer of indirection. This aims to make that a bit easier. llvm-svn: 369922
* [dotest] Don't set the DWARF version override in CFLAGS_EXTRA.Jonas Devlieghere2019-08-201-5/+4
| | | | | | | | We cannot override the DWARF version in the CFLAGS_EXTRA because they are used by tests that explicitly build without debug info. Instead, we pass them through the regular CFLAGS. llvm-svn: 369477
* [dotest] Add --dwarf-version to override the tested DWARF version.Jonas Devlieghere2019-08-191-2/+8
| | | | | | | | | | | | | On the matrix bot on GreenDragon [1] we want to run the test suite against different DWARF versions. The idea here is not to replace targeted tests for certain DWARF features, but rather to provide an easy way to support this configuration. [1] http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake-matrix/ Differential revision: https://reviews.llvm.org/D66370 llvm-svn: 369272
* Update Python tests for lldb-server on WindowsAaron Smith2019-08-141-2/+2
| | | | | | | | | | | | | | | | Summary: Thanks to Hui Huang and reviewers for all the help with this patch! Reviewers: labath, jfb, clayborg Reviewed By: labath Subscribers: Hui, clayborg, dexonsmith, lldb-commits Tags: #lldb Differential Revision: https://reviews.llvm.org/D61687 llvm-svn: 368776
* [dotest] Remove multiprocessingJonas Devlieghere2019-07-301-72/+4
| | | | | | | | | | | Now that the Xcode project is removed, I want to focus on dotest as a test framework, and remove its driver capabilities for which we already rely on llvm's lit. Removing multiprocessing is the first step in that direction. Differential revision: https://reviews.llvm.org/D65311 llvm-svn: 367331
* [dotest] Remove dead codeJonas Devlieghere2019-07-261-43/+2
| | | | | | Remove some dead code that I ran into when preparing D65311. llvm-svn: 367079
* [lldb][test_suite] skip tests of `libstdcpp` on Android and clean upAlex Langford2019-07-231-1/+3
| | | | | | | | | | | | | | | | Summary: Delete the android target from `libstdcpp` test category, since android no longer support libstdcxx Reviewers: xiaobai, labath Reviewed By: labath Subscribers: srhines, lldb-commits Tags: #lldb Differential Revision: https://reviews.llvm.org/D64771 llvm-svn: 366770
* [lldb][NFC] Cleanup mentions and code related to lldb-miRaphael Isemann2019-07-191-18/+1
| | | | | | | | | | | | | | | | Summary: lldb-mi has been removed, but there are still a bunch of references in the code base. This patch removes all of them. Reviewers: JDevlieghere, jfb Reviewed By: JDevlieghere Subscribers: dexonsmith, ki.stfu, mgorny, abidh, jfb, lldb-commits Tags: #lldb Differential Revision: https://reviews.llvm.org/D64992 llvm-svn: 366590
* [dotest] Add the ability to set environment variables for the inferior.Jonas Devlieghere2019-06-261-0/+3
| | | | | | | | | | | This patch adds a dotest flag for setting environment variables for the inferior. This is different from the current --env flag, which sets variables in the debugger's environment. This allows us to set things like LD_LIBRARY_PATH for testing. Differential revision: https://reviews.llvm.org/D63790 llvm-svn: 364443
* [lldb] [test] Watchpoint tests can be always run as root on NetBSDMichal Gorny2019-06-171-0/+2
| | | | llvm-svn: 363551
* [lldb] [test] Skip watchpoint tests on NetBSD if userdbregs is disabledMichal Gorny2019-06-171-0/+25
| | | | | | | | | | | Skip watchpoint tests if security.models.extensions.user_set_dbregs is disabled. This indicates that unprivileged processes are not allowed to write to debug registers which is a prerequisite for using hardware watchpoints. Differential Revision: https://reviews.llvm.org/D63380 llvm-svn: 363536
* Use list comprehension instead of map/filter to prepare Python2/3 compatSerge Guelton2019-03-211-2/+1
| | | | | | Differential Revision: https://reviews.llvm.org/D59579 llvm-svn: 356647
* Add ASAN llvm build directory variants toJason Molenda2019-03-081-0/+3
| | | | | | get_llvm_bin_dirs(). llvm-svn: 355661
* [Python] Update checkDsymForUUIDIsOn to be compatible with Python 3.Davide Italiano2019-01-101-4/+5
| | | | | | | | | | | | | | | | Summary: In python 2, strings and bytes are the same, but they're not in python 3, hence the return of read() needs an explicit conversion. While I'm around, rename the return of Popen() from `pipe` to `process`, as that's what Popen returns. Reviewers: JDevlieghere, friss, zturner, aprantl, serge-sans-paille Subscribers: lldb-commits Differential Revision: https://reviews.llvm.org/D56517 llvm-svn: 350788
* When no FileCheck binary is specified, look in the llvm/clang binJason Molenda2018-11-011-0/+32
| | | | | | | dirs relative to the source directory (Xcode build style) to find one, use it if found. llvm-svn: 345912
* [dotest] Make a missing FileCheck binary a warning, not an errorVedant Kumar2018-10-121-4/+8
| | | | | | | | | | This allows bots which haven't updated to pass in --filecheck to dotest.py to run more tests. FileCheck-dependent tests will continue to fail. Differential Revision: https://reviews.llvm.org/D53175 llvm-svn: 344401
* Allow use of self.filecheck in LLDB tests (c.f self.expect)Vedant Kumar2018-09-181-0/+7
| | | | | | | | | | | | | Add a "filecheck" method to the LLDB test base. This allows test authors to pattern match command output using FileCheck, making it possible to write stricter tests than what `self.expect` allows. For context (motivation, examples of stricter checking, etc), see the lldb-dev thread: "Using FileCheck in lldb inline tests". Differential Revision: https://reviews.llvm.org/D50751 llvm-svn: 342508
* [dotest] Make --test-subdir work with --no-multiprocessVedant Kumar2018-08-161-6/+11
| | | | | | | | | | | | | | | | | | | The single-process test runner is invoked in a number of different scenarios, including when multiple test dirs are specified or (afaict) when lit is used to drive the test suite. Unfortunately the --test-subdir option did not work with the single process test runner, breaking an important use case (using lit to run swift-lldb Linux tests): Failure URL: https://ci.swift.org/job/swift-PR-Linux/6841 We won't be able to run lldb tests within swift PR testing without filtering down the set of tests. This change makes --test-subdir work with the single-process runner. llvm-svn: 339929
* Add a new tool named "lldb-vscode" that implements the Visual Studio Code ↵Greg Clayton2018-08-161-0/+13
| | | | | | | | | | | | Debug Adaptor Protocol This patch adds a new lldb-vscode tool that speaks the Microsoft Visual Studio Code debug adaptor protocol. It has full unit tests that test all packets. This tool can be easily packaged up into a native extension and used with Visual Studio Code, and it can also be used by Nuclide Differential Revision: https://reviews.llvm.org/D50365 llvm-svn: 339911
* [testsuite] Implement a category to skip libstdcxx testsDavide Italiano2018-07-101-0/+18
| | | | | | | | | | | | | | | | On systems where it's not supported. As far as I understand Linux is the only systems which now ships with libstdcxx (maybe NetBSD?, but I'm not entirely sure of the state of lldb on the platform). We could make this more fine grained looking for the header as we do for libcxx. This is a little tricky as there's no such thing as /usr/include/c++/v1, but libstdcxx encodes the version number in the path (i.e. /usr/include/c++/5.4). I guess we might match a regex, but it seems fragile to me. Differential Revision: https://reviews.llvm.org/D49110 llvm-svn: 336724
* [lit, lldbsuite] Add a bug reference to the failing TestLinuxCore and fix an ↵Stella Stamenova2018-05-091-2/+2
| | | | | | | | | | | | | | | | | | undefined property in dotest.py Summary: 1) In TestLinuxCore rather than skipping the tests on Windows, mark them as expected failures and add a bug reference 2) In dotest.py replace the undefined property in the exceptions with the actual property causing the exception Reviewers: asmith, labath, zturner Reviewed By: labath, zturner Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D46590 llvm-svn: 331886
* Test Commit: fix a comment to be grammatically correctStella Stamenova2018-05-071-1/+1
| | | | llvm-svn: 331679
* [dotest] Make the set of tests independent of the test configurationPavel Labath2018-04-241-0/+17
| | | | | | | | | | | | | | | | | | | | | | | Summary: In the magic test duplicator, we were making the decision whether to create a test variant based on the compiler and the target platform. This meant that the set of known tests was different for each test configuration. This patch makes the set of generated test variants static and handles the skipping via runtime checks instead. This is more consistent with how we do other test-skipping decision (e.g. for libc++ tests), and makes it easier to expose the full set of tests to lit, which now does not need to know anything about what things can potentially cause tests to appear or disappear. Reviewers: JDevlieghere, aprantl Subscribers: eraman, lldb-commits Differential Revision: https://reviews.llvm.org/D45949 llvm-svn: 330708
* [dotest] Fix syntax error and typo.Jonas Devlieghere2018-04-121-1/+1
| | | | | | Python uses `elif` rather than `else if`. Fixes r329889. llvm-svn: 329890
* [dotest] Use in-tree dsymutil on DarwinJonas Devlieghere2018-04-121-0/+6
| | | | | | | | | | | | | | | | | | | Summary: With the upstream implementation of dsymutil containing almost all functionality from the one shipped with Xcode, we want to use the in-tree version for running the test suite. This will also allow us to re-enable TestUnicodeSymbols which was failing because of the discrepancy in how Unicode symbols were hashed in lldb and older versions of dsymutil. Reviewers: aprantl, davide, jingham, labath Subscribers: mgorny, llvm-commits, lldb-commits Differential Revision: https://reviews.llvm.org/D45518 llvm-svn: 329889
* [dotest] --skip-category should append and not override.Davide Italiano2018-04-051-1/+1
| | | | | | <rdar://problem/38566150> llvm-svn: 329358
* [test] Skip a test when using an out-of-tree debugserverVedant Kumar2018-03-081-0/+3
| | | | | | | | | | | | | | | | | | | | The test "test_fp_special_purpose_register_read" in TestRegisters.py fails on Darwin machines configured to use an out-of-tree debugserver. The error message is: 'register read ftag' returns expected result, got 'ftag = 0x80'. This indicates that the debugserver in use is too old. This commit introduces a decorator which can be used to skip tests which rely on having a just-built debugserver. This resolves the issue: $ ./bin/llvm-dotest -p TestRegisters.py -v 1 out of 617 test suites processed - TestRegisters.py Test Methods: 7 Success: 6 Skip: 1 ... llvm-svn: 327052
* Use an alternative approach to prevent Spotlight from indexing the build ↵Adrian Prantl2018-02-021-8/+0
| | | | | | directory. llvm-svn: 324115
* Create a marker for Spotlight to never index $BUILD_DIR.Adrian Prantl2018-02-011-2/+11
| | | | | | | | | | LLDB queries Spotlight to locate .dSYM bundles based on the UUID embedded in a binary, and because the UUID is a hash of filename and .text section, there *will* be conflicts inside $BUILD_DIR. This should fix the broken green dragon bots. llvm-svn: 324019
* Compile the LLDB tests out-of-tree.Adrian Prantl2018-01-301-0/+13
| | | | | | | | | | | | | | | | | | | | | | | | This patch is the result of a discussion on lldb-dev, see http://lists.llvm.org/pipermail/lldb-dev/2018-January/013111.html for background. For each test (should be eventually: each test configuration) a separate build directory is created and we execute make VPATH=$srcdir/path/to/test -C $builddir/path/to/test -f $srcdir/path/to/test/Makefile -I $srcdir/path/to/test In order to make this work all LLDB tests need to be updated to find the executable in the test build directory, since CWD still points at the test's source directory, which is a requirement for unittest2. Although we have done extensive testing, I'm expecting that this first attempt will break a few bots. Please DO NOT HESITATE TO REVERT this patch in order to get the bots green again. We will likely have to iterate on this some more. Differential Revision: https://reviews.llvm.org/D42281 llvm-svn: 323803
* [lldbtestsuite] Don't crash on `None` input for is_exe().Davide Italiano2018-01-261-0/+2
| | | | | | | | Now the function returns `False`, and the caller can take the appropriate action (in this case, for --executable, reporting that the file doesn't exist). llvm-svn: 323566
* [dotest] Remove crashinfo hookPavel Labath2018-01-101-1/+0
| | | | | | | | | | | | | | | | Summary: This used to be important when all tests were run in a single process, but that has no longer been the case for a while. Furthermore, this hook fails to build on new mac versions for several people, and it's not clear whether fixing it is worth the effort. Reviewers: jingham, clayborg, davide Subscribers: lldb-commits Differential Revision: https://reviews.llvm.org/D41871 llvm-svn: 322167
* Revert "[lldbtests] Handle errors instead of crashing."Pavel Labath2017-10-241-5/+1
| | | | | | | | | | | | | The commit breaks the case where you specify just a filename to the compiler. Previously, it would look up the compiler in your path, now it complains that the compiler is not found. One of the lldb buildbots is depending on this. It seems like a nice feature to have, as it means less typing and being able to avoid hard-coding the system compiler path in the bot config. This reverts commit r316393. llvm-svn: 316451
* [lldbtests] Handle errors instead of crashing.Davide Italiano2017-10-231-1/+5
| | | | | | | | | | | | | | | | | | If you pass an invalid compiler/debugger path on the cmdline to `dotest.py` this is what you get. Traceback (most recent call last): [...] File "dotest.py", line 7, in <module> lldbsuite.test.run_suite() [...] And with the patch applied: /home/davide/work/build-lldb/bin/clandasfasg is not a valid path, exiting Differential Revision: https://reviews.llvm.org/D39199 llvm-svn: 316393
OpenPOWER on IntegriCloud