summaryrefslogtreecommitdiffstats
path: root/lldb/packages/Python/lldbsuite/test/dotest.py
Commit message (Collapse)AuthorAgeFilesLines
* [dotest] Remove crashinfo hookPavel Labath2018-01-101-1/+0
| | | | | | | | | | | | | | | | Summary: This used to be important when all tests were run in a single process, but that has no longer been the case for a while. Furthermore, this hook fails to build on new mac versions for several people, and it's not clear whether fixing it is worth the effort. Reviewers: jingham, clayborg, davide Subscribers: lldb-commits Differential Revision: https://reviews.llvm.org/D41871 llvm-svn: 322167
* Revert "[lldbtests] Handle errors instead of crashing."Pavel Labath2017-10-241-5/+1
| | | | | | | | | | | | | The commit breaks the case where you specify just a filename to the compiler. Previously, it would look up the compiler in your path, now it complains that the compiler is not found. One of the lldb buildbots is depending on this. It seems like a nice feature to have, as it means less typing and being able to avoid hard-coding the system compiler path in the bot config. This reverts commit r316393. llvm-svn: 316451
* [lldbtests] Handle errors instead of crashing.Davide Italiano2017-10-231-1/+5
| | | | | | | | | | | | | | | | | | If you pass an invalid compiler/debugger path on the cmdline to `dotest.py` this is what you get. Traceback (most recent call last): [...] File "dotest.py", line 7, in <module> lldbsuite.test.run_suite() [...] And with the patch applied: /home/davide/work/build-lldb/bin/clandasfasg is not a valid path, exiting Differential Revision: https://reviews.llvm.org/D39199 llvm-svn: 316393
* Centralize libc++ test skipping logicPavel Labath2017-03-291-0/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This aims to replace the different decorators we've had on each libc++ test with a single solution. Each libc++ will be assigned to the "libc++" category and a single central piece of code will decide whether we are actually able to run libc++ test in the given configuration by enabling or disabling the category (while giving the user the opportunity to override this). I started this effort because I wanted to get libc++ tests running on android, and none of the existing decorators worked for this use case: - skipIfGcc - incorrect, we can build libc++ executables on android with gcc (in fact, after this, we can now do it on linux as well) - lldbutil.skip_if_library_missing - this checks whether libc++.so is loaded in the proces, which fails in case of a statically linked libc++ (this makes copying executables to the remote target easier to manage). To make this work I needed to split out the pseudo_barrier code from the force-included file, as libc++'s atomic does not play well with gcc on linux, and this made every test fail, even though we need the code only in the threading tests. So far, I am only annotating one of the tests with this category. If this does not break anything, I'll proceed to update the rest. Reviewers: jingham, zturner, EricWF Subscribers: srhines, lldb-commits Differential Revision: https://reviews.llvm.org/D30984 llvm-svn: 299028
* Fix remote test suite directory creationPavel Labath2017-03-201-2/+9
| | | | | | | | | | | | | | | r298203 make SBPlatform::MakeDirectory less recursive, which breaks the test suite creation of test directory hierarchy creation on the remote target. Since the function was never fully recursive, and the name does not imply recursiveness, I fix the problem by modifying the test runner to do the recursion manually. I also make the runner complain more loudly when it fails to create the directory -- previously it just printed the error to stdout and caused most of the tests to hang, which is not very helpful in diagnosing the problem. llvm-svn: 298261
* allow for specification of compiler/lldb executables basenameTim Hammerquist2017-03-171-0/+4
| | | | llvm-svn: 298123
* executables should be validated before spawning subprocessesTim Hammerquist2017-03-171-2/+14
| | | | | | | | | | | | | | dotest.py script doesn't validate executables passed on the command line before spawning dozens of subprocesses, all of which fail silently, leaving an empty results file. We should validate the lldb and compiler executables on configuration, aborting when given invalid paths, to prevent numerous, cryptic, and spurious failures. <rdar://problem/31117272> llvm-svn: 298111
* dotest.py: remove the ability to specify different architectures/compilers ↵Pavel Labath2017-03-151-128/+87
| | | | | | | | | | | | | | | | | | | | | | | | | in a single invocation Summary: This has been broken at least since the new test result framework was added, which was over a year ago. It looks like nobody has missed it since. Removing this makes the gmodules handling code saner, as it already did not know how to handle the multiple-compilers case. My motivation for this is libc++ data formatters support on android -- I am trying make a central way of determining whether libc++ tests can be run, and without this, I would have to resort to similar hacks as the gmodules code. Reviewers: jingham, zturner Subscribers: danalbert, tfiala, lldb-commits Differential Revision: https://reviews.llvm.org/D30779 llvm-svn: 297811
* [CMake] Override debugserver to use the build tree on DarwinChris Bieneman2017-03-141-0/+3
| | | | | | This patch adds support to the test suite for overriding the path to debugserver, and uses the override to point to the build tree's debugserver on Darwin. llvm-svn: 297776
* [Test-Suite] Speculative fix for darwin botsChris Bieneman2016-10-311-2/+2
| | | | | | When building with Xcode it looks like some of the logic in my test changes went haywire. This should fix it. llvm-svn: 285646
* [Test Suite] Properly respect --framework optionChris Bieneman2016-10-311-0/+3
| | | | | | | | | | | | | | | Summary: dotest.py has a framework option that is not respected. This patch makes the framework path properly configurable via the --framework option. This patch also adds a function to the lldbtest.Base class named "hasDarwinFramework" which allows us to not rely on the host platform to determine if a framework is present. If running on Darwin, and not building a framework, this will follow the *nix code paths which are appropriate for Darwin. Reviewers: tfiala Subscribers: lldb-commits, mgorny Differential Revision: https://reviews.llvm.org/D25886 llvm-svn: 285541
* [Test Suite] Allow overriding codesign identityChris Bieneman2016-10-211-0/+2
| | | | | | | | | | | | Summary: Not everyone names their code sign identity "lldb_codesign", so it is nice to allow this to be overridden. Reviewers: zturner, tfiala Subscribers: labath, mgorny, lldb-commits Differential Revision: https://reviews.llvm.org/D25714 llvm-svn: 284893
* Fix lookup path for lldb-miChris Bieneman2016-10-121-6/+7
| | | | | | | | | | | | | | | | | Summary: The test suite calls realpath on the lldb executable then append "-mi" to it to find the path of the lldb-mi executable. This does not work when using CMake builds on *nix platforms. On *nix platforms when a version number is set on executables CMake generates the binary as ${name}-${version} with a symlink named ${name} pointing to it. This results in the lldb executable being named lldb-4.0.0, and since lldb-4.0.0-mi doesn't ever match the lldb-mi executable these tests are always disabled. This patch looks for lldb-mi in the same directory as lldb. Reviewers: zturner, tfiala Subscribers: ki.stfu, enlight, lldb-commits Differential Revision: https://reviews.llvm.org/D25486 llvm-svn: 284041
* Improvements to testing blacklistFrancis Ricci2016-10-041-22/+14
| | | | | | | | | | | | | | | | | | | Summary: This patch is necessary because individual test cases are not required to have unique names. Therefore, test cases must now be specified explicitly in the form <TestCase>.<TestMethod>. Because it works by regex matching, passing just <TestCase> will still disable an entire file. This also allows for multiple exclusion files to be specified. Reviewers: zturner, labath, jingham, tfiala Subscribers: lldb-commits, sas Differential Revision: https://reviews.llvm.org/D24988 llvm-svn: 283238
* Allow for tests to be disabled at runtimeFrancis Ricci2016-09-231-1/+51
| | | | | | | | | | | | | | | | Summary: The current implementation of the test suite allows the user to run a certain subset of tests using '-p', but does not allow the inverse, where a user wants to run all but some number of known failing tests. Implement this functionality. Reviewers: labath, zturner, tfiala Subscribers: jingham, sas, lldb-commits Differential Revision: https://reviews.llvm.org/D24629 llvm-svn: 282298
* *** This commit represents a complete reformatting of the LLDB source codeKate Stone2016-09-061-105/+240
| | | | | | | | | | | | | | | | | | | | | | | *** to conform to clang-format’s LLVM style. This kind of mass change has *** two obvious implications: Firstly, merging this particular commit into a downstream fork may be a huge effort. Alternatively, it may be worth merging all changes up to this commit, performing the same reformatting operation locally, and then discarding the merge for this particular commit. The commands used to accomplish this reformatting were as follows (with current working directory as the root of the repository): find . \( -iname "*.c" -or -iname "*.cpp" -or -iname "*.h" -or -iname "*.mm" \) -exec clang-format -i {} + find . -iname "*.py" -exec autopep8 --in-place --aggressive --aggressive {} + ; The version of clang-format used was 3.9.0, and autopep8 was 1.2.4. Secondly, “blame” style tools will generally point to this commit instead of a meaningful prior commit. There are alternatives available that will attempt to look through this change and find the appropriate prior commit. YMMV. llvm-svn: 280751
* Allow custom formatting of session log file names.Zachary Turner2016-05-171-0/+1
| | | | | | Differential Revision: http://reviews.llvm.org/D20306 llvm-svn: 269793
* test: remove use of undefined variablesSaleem Abdulrasool2016-05-161-2/+1
| | | | | | | | The variables referenced in the print message are not defined. Simply state that the requisite script is not found. Correct grammar to indicate that the tests are rather likely to fail rather than unlikely to fail. llvm-svn: 269628
* test: add missing splatSaleem Abdulrasool2016-05-161-1/+1
| | | | | | | | The parameter here is a list, not a string. Ensure that the we splat the list into arguments prior to invoke os.path.join. This would previously fail with a `startswith` is not a member of `list`. llvm-svn: 269627
* test: add missing parameterSaleem Abdulrasool2016-05-161-1/+1
| | | | | | | Add the missing required parameter to the function. This permits tests to get a bit further before failing. llvm-svn: 269626
* test infra: catch bad decorators and import-time errorsTodd Fiala2016-05-131-54/+79
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This change enhances the LLDB test infrastructure to convert load-time exceptions in a given Python test module into errors. Before this change, specifying a non-existent test decorator, or otherwise having some load-time error in a python test module, would not get flagged as an error. With this change, typos and other load-time errors in a python test file get converted to errors and reported by the test runner. This change also includes test infrastructure tests that include covering the new work here. I'm going to wait until we have these infrastructure tests runnable on the main platforms before I try to work that into all the normal testing workflows. The test infrastructure tests can be run by using the standard python module testing practice of doing the following: cd packages/Python/lldbsuite/test_event python -m unittest discover -s test/src -p 'Test*.py' Those tests run the dotest inferior with a known broken test and verify that the errors are caught. These tests did not pass until I modified dotest.py to capture them properly. @zturner, if you have the chance, if you could try those steps above (the python -m unittest ... line) on Windows, that would be great if we can address any python2/3/Windows bits there. I don't think there's anything fancy, but I didn't want to hook it into test flow until I know it works there. I'll be slowly adding more tests that cover some of the other breakage I've occasionally seen that didn't get collected as part of the summarization. This is the biggest one I'm aware of. Reviewers: zturner, labath Subscribers: zturner, lldb-commits Differential Revision: http://reviews.llvm.org/D20193 llvm-svn: 269489
* fix argument usage for '-#' command line optionTodd Fiala2016-05-051-1/+1
| | | | | | | This was broken in the grand configuration change. Now using -# works again. llvm-svn: 268638
* Store absolute path for lldb executable in dotest.pyFrancis Ricci2016-04-251-1/+1
| | | | | | | | | | | | | | | | | | | | Summary: lldb-server tests are currently being skipped on the check-lldb target. This is because we get the path of lldb-server by modifying the path to the lldb executable. However, by this point, we've changed directories, and a relative path to the build/bin directory will no longer point to the location of lldb-server. Storing an absolute path solves this issue. Reviewers: vharron, zturner, tfiala, labath Subscribers: labath, lldb-commits, sas Differential Revision: http://reviews.llvm.org/D19082 llvm-svn: 267463
* test infra: move test event-related handling into its own packageTodd Fiala2016-04-201-10/+8
| | | | | | | | | | | | | | | | | | | | | | | This change moves all the test event handling and its related ResultsFormatter classes out of the packages/Python/lldbsuite/test dir into a packages/Python/lldbsuite/test_event package. Formatters are moved into a sub-package under that. I am limiting the scope of this change to just the motion and a few minor issues caught by a static Python checker (e.g. removing unused import statements). This is a pre-step for adding package-level tests to the test event system. I also intend to simplify test event results formatter selection after I make sure this doesn't break anybody. See: http://reviews.llvm.org/D19288 Reviewed by: Pavel Labath llvm-svn: 266885
* Unicode support on Win32.Zachary Turner2016-03-221-1/+1
| | | | | | | | | | | | | Win32 API calls that are Unicode aware require wide character strings, but LLDB uses UTF8 everywhere. This patch does conversions wherever necessary when passing strings into and out of Win32 API calls. Patch by Cameron Differential Revision: http://reviews.llvm.org/D17107 Reviewed By: zturner, amccarth llvm-svn: 264074
* Eliminate the TestStarted-XXX and TestFinished-XXX files from check-lldb runs.Adrian McCarthy2016-03-101-11/+0
| | | | | | | | Nobody seems to know what purpose these files serve, yet they were accumulating by the thousands in the test traces directory. I'm proposing we delete them. Creating these files accounted for about 2.5% of the time to run ninja check-lldb on my machine, which isn't a lot, but it's something. llvm-svn: 263122
* Add a symbolic link from the test directory to the actual - elsewhere ↵Enrico Granata2015-12-161-1/+1
| | | | | | | | | | located - path that contains the test cases - and teach the test suite driver to resolve paths that contain symbolic links to test cases This is meant to reduce the typing that one needs to do to get from the test subdirectory to actual test cases. Now one can just do $ ./dotest.py ./testcases/<yaddayaddayadda> llvm-svn: 255741
* test-infra: refactored new summary results into base ResultsFormatter classTodd Fiala2015-12-151-1/+1
| | | | | | | This allows more specialized formatters to still reuse the results summarization display from the base class. llvm-svn: 255676
* test infra: enable single-worker rerun phase for flakey tests.Todd Fiala2015-12-141-0/+1
| | | | | | | | | | | | | | | | Use of --rerun-all-issues will enable any test method failure, not just test methods marked with the flakey decorator, to rerun. Currently this does not change the flakey logic's immediate rerun attempt. I want to make sure this doesn't cause any significant issues before changing that part. The rerun reporting is only known to work properly with the default (new) BasicResultsFormatter reporting. Once we work out any issues, I'll go back and make sure the curses output handles it properly as well. llvm-svn: 255543
* Make debug info specification use categories system.Zachary Turner2015-12-141-17/+0
| | | | | | | Reviewed By: Tamas Berghammer, Pavel Labath Differential Revision: http://reviews.llvm.org/D15428 llvm-svn: 255525
* test infra: adds book-keeping for rerunnable testsTodd Fiala2015-12-121-0/+3
| | | | | | | | | | | | | | Also adds full path info for exceptional exits and timeouts when no test method is currently running. Adds --rerun-all-issues command line arg. If specified, all test issues are eligible for rerun. If not specified, only tests marked flakey are eligible for rerun. The actual rerunning will occur in an upcoming change. This change just handles tha accounting of what should be rerun. llvm-svn: 255438
* Revert "Revert "Turn on new test summary results by default.""Todd Fiala2015-12-111-0/+6
| | | | | | This reverts commit f994b46a2028c8a8b9b55fe010a95122bca07540. llvm-svn: 255381
* Remove -S option from dotest.py.Zachary Turner2015-12-111-3/+0
| | | | llvm-svn: 255361
* Remove -r and -R options from dotest.py.Zachary Turner2015-12-111-100/+1
| | | | llvm-svn: 255360
* Revert "Turn on new test summary results by default."Pavel Labath2015-12-111-6/+0
| | | | | | | The new test summary formatter does not honor the "expected timeout" markings, which makes our buildbots all red. I'm switching it off by default until we figure out a way to make this work. llvm-svn: 255335
* Remove the -y option from dotest.py.Zachary Turner2015-12-101-4/+0
| | | | llvm-svn: 255280
* Remove the -x option from dotest.py.Zachary Turner2015-12-101-5/+0
| | | | llvm-svn: 255279
* Remove deprecated command line options from dotest.pyZachary Turner2015-12-101-10/+0
| | | | llvm-svn: 255278
* Remove the --output-on-success command line argument from dotest.Zachary Turner2015-12-101-7/+2
| | | | llvm-svn: 255277
* Remove the -T option from dotest.py.Zachary Turner2015-12-101-13/+0
| | | | llvm-svn: 255276
* Remove -w option from dotest.py.Zachary Turner2015-12-101-3/+0
| | | | llvm-svn: 255275
* Remove the -P option from dotest.pyZachary Turner2015-12-091-7/+0
| | | | | | | | This was an option to display a graphical progress bar. Nobody is using this, and it doesn't work correctly anyway with the new result formatter. llvm-svn: 255153
* Delete the -n command line option from dotest.py.Zachary Turner2015-12-091-14/+7
| | | | | | | This removes the option to not print some one time version and diagnostic information when running the test suite. llvm-svn: 255152
* Remove the -i command line option from dotest.py.Zachary Turner2015-12-091-7/+0
| | | | | | | This is part of a larger effort to remove unused command line options from dotest.py. llvm-svn: 255151
* Delete the -F command line option from dotest.py.Zachary Turner2015-12-091-5/+0
| | | | | | | | This removes the failfast command line option as part of an effort to simplify dotest and remove unused command line options. You can still Ctrl+C any time you want to exit early. llvm-svn: 255150
* Remove -k command line option from dotest.py.Zachary Turner2015-12-091-3/+0
| | | | | | This is part of an effort to remove unused command line options. llvm-svn: 255143
* Remove -e option from dotest.py.Zachary Turner2015-12-091-7/+0
| | | | | | | This is part of an effort to clean up dotest command line options that are no longer used. llvm-svn: 255142
* Remove -libcxx option from dotest.pyZachary Turner2015-12-091-3/+0
| | | | | | | Nobody was using this, and plus it can be achieved just as well by using -E to set an environment variable. llvm-svn: 255141
* Turn on new test summary results by default.Todd Fiala2015-12-091-0/+6
| | | | llvm-svn: 255130
* wire timeouts and exceptional inferior process exits through the test event ↵Todd Fiala2015-12-091-2/+11
| | | | | | | | | | | | | | | | | | | | | | | system The results formatter system is now fed timeouts and exceptional process exits (i.e. inferior dotest.py process that exited by signal on POSIX systems). If a timeout or exceptional exit happens while a test method is running on the worker queue, the timeout or exceptional exit is charged and reported against that test method. Otherwise, if no test method was running at the time of the timeout or exceptional exit, only the test filename will be reported as the TIMEOUT or ERROR. Implements: https://llvm.org/bugs/show_bug.cgi?id=24830 https://llvm.org/bugs/show_bug.cgi?id=25703 In support of: https://llvm.org/bugs/show_bug.cgi?id=25450 llvm-svn: 255097
OpenPOWER on IntegriCloud