summaryrefslogtreecommitdiffstats
path: root/lldb/packages/Python/lldbsuite/test/lldbtest.py
Commit message (Collapse)AuthorAgeFilesLines
* test infra: move test event-related handling into its own packageTodd Fiala2016-04-201-4/+3
| | | | | | | | | | | | | | | | | | | | | | | This change moves all the test event handling and its related ResultsFormatter classes out of the packages/Python/lldbsuite/test dir into a packages/Python/lldbsuite/test_event package. Formatters are moved into a sub-package under that. I am limiting the scope of this change to just the motion and a few minor issues caught by a static Python checker (e.g. removing unused import statements). This is a pre-step for adding package-level tests to the test event system. I also intend to simplify test event results formatter selection after I make sure this doesn't break anybody. See: http://reviews.llvm.org/D19288 Reviewed by: Pavel Labath llvm-svn: 266885
* Retry deletion of temporary files to avoid race conditions on Windows.Adrian McCarthy2016-04-111-12/+15
| | | | | | Differential Revision: http://reviews.llvm.org/D18912 llvm-svn: 265948
* [NFC] Cleanup the code used to run shell commands from testsTamas Berghammer2016-04-051-0/+6
| | | | | | | | | | | | Previously we had 3 different method to run shell commands on the target and 4 copy of code waiting until a given file appears on the target device (used for syncronization). This CL merges these methods to 1 run_platform_command and 1 wait_for_file_on_target functions located in some utility classes. Differential revision: http://reviews.llvm.org/D18789 llvm-svn: 265398
* Don't vary debug info for lldb-server testsPavel Labath2016-03-311-1/+11
| | | | | | | | | | | | | | | | | | Summary: Debug info is used only by the client and lldb-server tests do not even have the client component running, as they communicate with the server directly. Therefore, running the tests for each debug info type is unnecessarry. This adds general ability to mark a test class as not dependent on debug info, and marks all lldb-server tests as such. Reviewers: tberghammer, tfiala Subscribers: lldb-commits Differential Revision: http://reviews.llvm.org/D18598 llvm-svn: 265017
* Made self.expect() errors a little more readable in the testsuite.Sean Callanan2016-03-081-7/+8
| | | | | | | | | | | | | | | | | self.expect() had two problems: - If there was a substrs argument, then it overwrote the variable containing the command to run with the last substr. That meant nonsense command text in testsuite errors. - The actual output is not printed, which makes fixing testsuite failures a bit annoying (you end up having to use the -tv arguments to dotest). This fixes both of these issues. We could do even better, pretty-printing the criteria for "correct" output, but this at least makes dealing with errors a bit better. llvm-svn: 262950
* Adjust for Python-3.Siva Chandra2016-02-121-0/+4
| | | | | | | | | | | | | | | Summary: This does not yet give us a clean testsuite run but it does help with: 1. Actually building on linux 2. Run the testsuite with over 70% tests passing on linux. Reviewers: tfiala, labath, zturner Subscribers: lldb-commits Differential Revision: http://reviews.llvm.org/D17182 llvm-svn: 260721
* Move the rest of the tests over to using the new decorator module.Zachary Turner2016-02-041-443/+7
| | | | llvm-svn: 259838
* Move some of the common decorators to decorators.py.Zachary Turner2016-02-041-247/+30
| | | | | | | | | | | | This doesn't attempt to move every decorator. The reason for this is that it requires touching every single test file to import decorators.py. I would like to do this in a followup patch, but in the interest of keeping the patches as bite-sized as possible, I've only attempted to move the underlying common decorators first. A few tests call these directly, so those tests are updated as part of this patch. llvm-svn: 259807
* Move some android platform functions to lldbplatformutil.Zachary Turner2016-02-031-73/+14
| | | | | | | | | | | | | | | | | | | My eventual goal is to move all of the test decorators to their own module such as `decorators.py`. But some of the decorators use existing functions in `lldbtest.py` and conceptually the functions are probably more appropriately placed in lldbplatformutil. Moreover, lldbtest.py is a huge file with a ton of random utility functions scattered around, so this patch also workds toward the goal of reducing the footprint of this one module to a more reasonable size. So this patch moves some of them over to lldbplatformutil with the eventual goal of moving decorators over to their own module. Reviewed By: Tamas Berghammer, Pavel Labath Differential Revision: http://reviews.llvm.org/D16830 llvm-svn: 259680
* Remove skipUnlessListedRemote test decoratorPavel Labath2016-02-031-12/+0
| | | | | | | | | | | | | | This decorator was used in only one test, and it's behaviour was quite complicated. It skipped if: - test was remote - platform was *not* android I am not aware of anyone running tests with this configuration (and even then, I am not aware of a reason why the test should not pass), but if TestLoadUnload starts breaking for you after this commit, please disable the test with @expectedFailureAll(remote=True, oslist=[YOUR_PLATFORM]) llvm-svn: 259642
* Don't return a tuple from the skip test function.Zachary Turner2016-02-021-31/+33
| | | | | | | | | | | Previously we were returning a tuple of (bool, skip_reason) from the tuple function. This makes for some awkward code, especially since a value of True for the first argument implies that the second argument is None, and a value of False implies that the second argument is not None. So it was basically redundant, and with this patch we simply return the skip reason or None directly. llvm-svn: 259590
* Re-write many skip decorators to use shared code.Zachary Turner2016-02-021-205/+90
| | | | | | | | | | | | | This should be no functional change, just a refactoring of the skip decorators to all centralize on a single function, `skipTestIfFn` that does all the logic. This allows easier maintenance of the decorators and also centralizes all the hard-to-understand logic in one place. Reviewed by: Pavel Labath Differential Revision: http://reviews.llvm.org/D16741 llvm-svn: 259543
* Fix compiler lookup when specified without pathPavel Labath2016-02-021-2/+1
| | | | | | | | | r259433 introduced a regression, where if a compiler is specified without a path (e.g., CC=clang, relying on the fact that clang is in $PATH), then the test suite would fail (at the compiler version detection step) because realpath would interpret this as a path relative to cwd). The fix is to perform the $PATH expansion (via `which`) before the realpath step. llvm-svn: 259484
* Always write the session file in UTF-8.Zachary Turner2016-02-011-1/+3
| | | | | | | | | | | | | | | | | | This patch attempts to solve the Python 2 / Python 3 incompatibilities by introducing a new `encoded_file` abstraction that we use instead of `io.open()`. The problem with the builtin implementation of `io.open` is that `read` and `write` accept and return `unicode` objects, which are not always convenient to work with in Python 2. We solve this by making `encoded_file.open()` return the same object returned by `io.open()` but with hooked `read()` and `write()` methods. These hooked methods will accept binary or text data, and conditionally convert what it gets to a `unicode` object using the correct encoding. When calling `read()` it also does any conversion necessary to convert the output back into the native `string` type of the running python version. Differential Revision: http://reviews.llvm.org/D16736 llvm-svn: 259379
* Revert "Resubmit r258759 with proper unicode handling."Zachary Turner2016-01-271-2/+2
| | | | | | This reverts commit 2c79d60214e146b13b233392a859b4f79340e90e. llvm-svn: 258978
* Resubmit r258759 with proper unicode handling.Zachary Turner2016-01-271-2/+2
| | | | | | | | | Instead of opening the file in unicode mode, we need only encode data which potentially has non-ASCII characters as UTF8 before writing. This should work across both Python versions, and is also far simpler than anything else discussed. llvm-svn: 258969
* Fix some python 3 incompatibilities that went in overnight.Zachary Turner2016-01-271-1/+1
| | | | | | | | * basestring is not a thing anymore. Must use `six.string_types`. * Must use from __future__ import print_function in every new test file. llvm-svn: 258967
* Refactor some of the xfail / skip decorators to share logic.Zachary Turner2016-01-271-71/+117
| | | | | | | | | | | Previously the logic of skipIf and expectedFailure were 99% the same, but they took different sets of arguments since they were maintained separately, and had slightly differences in their behavior. This makes everything consistent, there is now only one real implementation, and the previous ones are changed to use the single master implementation. llvm-svn: 258966
* [LLDB][MIPS] A small fix in GetBreakableLoadAddress() for MIPSBhushan D. Attarde2016-01-271-4/+11
| | | | | | | | | | | | | SUMMARY: Get the load address for the address given by symbol and function. Earlier, this was done for function only, this patch does it for symbol too. This patch also adds TestAvoidBreakpointInDelaySlot.py to test this change. Reviewers: clayborg Subscribers: labath, zturner, mohit.bhakkad, sagar, jaydeep, lldb-commits Differential Revision: http://reviews.llvm.org/D16049 llvm-svn: 258919
* Reverting r258759 as it is breaking the OSX buildEnrico Granata2016-01-261-1/+1
| | | | llvm-svn: 258791
* Write the session log file in UTF-8.Zachary Turner2016-01-261-1/+1
| | | | | | | | | | | Previously we were writing in the default encoding, which depends on the operating system and is not guaranteed to be unicode aware. On Python 3, this would lead to a situation where writing unicode text to the log file generates an exception. The fix here is to write session logs using the proper encoding, which incidentally fixes another test, so xfail is removed from that. llvm-svn: 258759
* Decode files with UTF-8 in lldbutil.line_number.Zachary Turner2016-01-221-1/+2
| | | | | | | | Since Unicode support is different in Py2 and Py3, Py3 was throwing exceptions about being unable to decode the file with the default encoding. llvm-svn: 258588
* Revert "Enable test log collection from remote debug servers"Pavel Labath2016-01-221-18/+6
| | | | | | | | Unfortunately, this turns out not to be working on the lldb-server tests, as there the server is started in a different way. Since this was a bit of a hack to start with, I am removing it until I can solve the problem more holistically. llvm-svn: 258501
* Enable test log collection from remote debug serversPavel Labath2016-01-211-6/+18
| | | | | | | | | | | | | | | | | Summary: We already have the ability to collect the server logs when doing local debugging. This enables the collection of remote logs as well. This relies on specifying a relative path "server.log" for LLDB_DEBUGSERVER_LOG_FILE when starting remote platform. Since we always set the platform working directory to a fresh folder to avoid conflicts, the actual file path will always be different and we can pick the logs up from there. Reviewers: tfiala Subscribers: lldb-commits Differential Revision: http://reviews.llvm.org/D16322 llvm-svn: 258414
* Remove skipIfLinuxClang decoratorPavel Labath2016-01-181-18/+0
| | | | | | it isn't used in the code anymore, and we're trying to cut down on the decorators. llvm-svn: 258049
* Guard against application of an XFAIL decorator on a classPavel Labath2016-01-181-0/+2
| | | | | | | | This does not work and causes the class to be silently skipped, which is a bad idea. This makes sure it cannot happen accidentaly. I've played with the idea of actually making the decorator work at class level, but it proved too magic to do at this moment. llvm-svn: 258048
* Fix detection of teardown hooks that are already bound.Adrian McCarthy2016-01-141-1/+1
| | | | llvm-svn: 257829
* Xfail some Arm-Linux specific failuresOmair Javaid2016-01-111-3/+4
| | | | | | | | | | Updated expectedFailureLinux decorator to reflect architecture Marked some triaged failures as xfails on arm with updated expectedFailureLinux decorator Differential revision: http://reviews.llvm.org/D15893 llvm-svn: 257405
* Remove old flaky test rerun logicPavel Labath2016-01-051-23/+3
| | | | | | | | | | | | | | Summary: This removes the old logic for rerunning flaky tests. The new test runners will take care of rerunning failing tests. Reviewers: tfiala Subscribers: lldb-commits Differential Revision: http://reviews.llvm.org/D15855 llvm-svn: 256824
* Allow test decorators to use lists (and not_in(list)) for archesAdrian McCarthy2015-12-221-3/+8
| | | | | | Differential Revision: https://mail.google.com/mail/u/0/?zx=w4areffgjbgg#inbox/151cb6afe6169bb0 llvm-svn: 256283
* [test] Add ability to expect timeoutsPavel Labath2015-12-161-0/+4
| | | | | | | | | | | | | | | | | Summary: This adds ability to mark test that do not complete due to hangs, crashes, etc., as "expected", to avoid flagging the build red for a known problem. Functionally, this extends the scope of the existing expectedFailureXXX decorators to cover these states as well. Once this is in, I will start replacing the magic list of failing tests in dosep.py with our regular annotations which should hopefully make code simpler. Reviewers: tfiala Subscribers: lldb-commits Differential Revision: http://reviews.llvm.org/D15530 llvm-svn: 255763
* Make few adjustments after r255542.Siva Chandra2015-12-151-3/+3
| | | | | | | | | | Reviewers: zturner Subscribers: lldb-commits Differential Revision: http://reviews.llvm.org/D15511 llvm-svn: 255584
* Remove the multiplier loop.Zachary Turner2015-12-141-7/+27
| | | | | | | | This is leading to some kind of subtle issue related to local functions and closures, so let's just go back to the old way for now. llvm-svn: 255567
* Make skipIf decorator support not_in() functor.Zachary Turner2015-12-141-6/+8
| | | | llvm-svn: 255542
* Make debug info specification use categories system.Zachary Turner2015-12-141-68/+20
| | | | | | | Reviewed By: Tamas Berghammer, Pavel Labath Differential Revision: http://reviews.llvm.org/D15428 llvm-svn: 255525
* Make test categories composablePavel Labath2015-12-141-2/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Previously the add_test_categories would simply overwrite the current set of categories for a method. This change makes the decorator truly "add" categories, by extending the current set of categories instead of replacing it. To do this, I have: - replaced the getCategories() property on a method (which was itself a method), with a simple list property "categories". This makes add_test_categories easier to implement, and test categories isn't something which should change between calls anyway. - rewritten the getCategoriesForTest function to merge method categories with the categories of the test case. Previously, it would just use the method categories if they were present. I have also greatly simplified this method. Originally, it would use a lot of introspection to enable it being called on various types of objects. Based on my tests, it was only ever being called on a test case. The new function uses much less introspection then the preivous one, so we should easily catch any stray uses, if there are any, as they will generate exceptions now. Reviewers: zturner, tfiala, tberghammer Subscribers: lldb-commits Differential Revision: http://reviews.llvm.org/D15451 llvm-svn: 255493
* Remove -S option from dotest.py.Zachary Turner2015-12-111-13/+1
| | | | llvm-svn: 255361
* Add test event marking a test as explicitly eligible for rerun if it is ↵Todd Fiala2015-12-111-0/+8
| | | | | | | | | | | | | | marked flakey. This will be used in a future change to support rerunning flakey tests that hit a test result isue in a low-load, single worker test runner phase. This is implemented as an additive-style event rather than being evaluated and added to the start_test event because the decorator code only runs after the start_test event is created and sent. i.e. LLDBTestResult.startTest() runs before the test method decorators run. llvm-svn: 255351
* Add NetBSD support in the buildDriver and buildLibrary routinesKamil Rytarowski2015-12-101-2/+2
| | | | | | | | | | | | Summary: NetBSD is like FreeBSD and Linux in these routines. Reviewers: clay.chang, tfiala, emaste, joerg Subscribers: lldb-commits, emaste Differential Revision: http://reviews.llvm.org/D15374 llvm-svn: 255308
* Remove -w option from dotest.py.Zachary Turner2015-12-101-12/+0
| | | | llvm-svn: 255275
* Remove -k command line option from dotest.py.Zachary Turner2015-12-091-24/+0
| | | | | | This is part of an effort to remove unused command line options. llvm-svn: 255143
* XFAIL TestReturnValue for remote Windows->Android testsPavel Labath2015-12-091-1/+3
| | | | | | this also adds the ability to match the host platform to the expectedFailureAll decorator. llvm-svn: 255105
* Remove the -c option from dotest.py.Zachary Turner2015-12-081-63/+1
| | | | | | | | | | This seems to be a legacy relic from days gone by where the remote test suite runner operated completely differently than it does today. git blames and comments traced this functionality back to about 2012, and nobody seems to know anything about it now. llvm-svn: 255060
* Remove the -b option from dotest.pyZachary Turner2015-12-081-12/+0
| | | | | | | This removes the blacklist option as part of an effort to remove unused / unmaintained command line options from the test suite. llvm-svn: 255040
* Remove +b option from dotest.pyZachary Turner2015-12-081-15/+1
| | | | llvm-svn: 255037
* Modify "platform connect" to connect to processes as wellTamas Berghammer2015-12-081-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | The standard remote debugging workflow with gdb is to start the application on the remote host under gdbserver (e.g.: gdbserver :5039 a.out) and then connect to it with gdb. The same workflow is supported by debugserver/lldb-gdbserver with a very similar syntax but to access all features of lldb we need to be connected also to an lldb-platform instance running on the target. Before this change this had to be done manually with starting a separate lldb-platform on the target machine and then connecting to it with lldb before connecting to the process. This change modifies the behavior of "platform connect" with automatically connecting to the process instance if it was started by the remote platform. With this command replacing gdbserver in a gdb based worflow is usually as simple as replacing the command to execute gdbserver with executing lldb-platform. Differential revision: http://reviews.llvm.org/D14952 llvm-svn: 255016
* Get rid of global variables in dotest.pyZachary Turner2015-12-081-39/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | This moves all the global variables into a separate module called `configuration`. This has a number of advantages: 1. Configuration data is centrally maintained so it's easy to get a high level overview of what configuration data the test suite makes use of. 2. The method of sharing configuration data among different parts of the test suite becomes standardized. Previously we would put some things into the `lldb` module, some things into the `lldbtest_config` module, and some things would not get shared. Now everything is shared through one module and is available to the entire test suite. 3. It opens the door to moving some of the initialization code into the `configuration` module, simplifying the implementation of `dotest.py`. There are a few stragglers that didn't get converted over to using the `configuration` module in this patch, because it would have grown the size of the patch unnecessarily. This includes everything currently in the `lldbtest_config` module, as well as the `lldb.remote_platform` variable. We can address these in the future. llvm-svn: 254982
* Add initial NetBSD support in lldbsuite/test/lldbtest.pyKamil Rytarowski2015-12-071-5/+21
| | | | | | | | | | | | | | | | | | | | | | | | Summary: Add new functions: - expectedFailureNetBSD() - expectedFlakeyNetBSD() - skipIfNetBSD() Add new NetBSD entry in: - getPlatform() - getHostPlatform() Assume that libc++ is installed and use the GNU toolchain Reviewers: joerg, emaste, tfiala, clayborg Subscribers: lldb-commits Differential Revision: http://reviews.llvm.org/D15262 llvm-svn: 254948
* Define new builder_netbsdKamil Rytarowski2015-12-051-0/+2
| | | | | | | | | | | | Summary: This is used in tests. Reviewers: emaste, tfiala, clayborg Subscribers: zturner, lldb-commits, joerg Differential Revision: http://reviews.llvm.org/D15065 llvm-svn: 254853
* Revert "Make skipIf support the not_in function (second attempt)."Siva Chandra2015-11-201-15/+11
| | | | | | | | | | | | Summary: This reverts commit 70dca28976ee8137acce2cc203dd394f4d761276. Reviewers: amccarth, zturner Subscribers: lldb-commits Differential Revision: http://reviews.llvm.org/D14881 llvm-svn: 253704
OpenPOWER on IntegriCloud