summaryrefslogtreecommitdiffstats
path: root/lldb/packages/Python/lldbsuite/test/lldbtest.py
Commit message (Collapse)AuthorAgeFilesLines
* Xfail some Arm-Linux specific failuresOmair Javaid2016-01-111-3/+4
| | | | | | | | | | Updated expectedFailureLinux decorator to reflect architecture Marked some triaged failures as xfails on arm with updated expectedFailureLinux decorator Differential revision: http://reviews.llvm.org/D15893 llvm-svn: 257405
* Remove old flaky test rerun logicPavel Labath2016-01-051-23/+3
| | | | | | | | | | | | | | Summary: This removes the old logic for rerunning flaky tests. The new test runners will take care of rerunning failing tests. Reviewers: tfiala Subscribers: lldb-commits Differential Revision: http://reviews.llvm.org/D15855 llvm-svn: 256824
* Allow test decorators to use lists (and not_in(list)) for archesAdrian McCarthy2015-12-221-3/+8
| | | | | | Differential Revision: https://mail.google.com/mail/u/0/?zx=w4areffgjbgg#inbox/151cb6afe6169bb0 llvm-svn: 256283
* [test] Add ability to expect timeoutsPavel Labath2015-12-161-0/+4
| | | | | | | | | | | | | | | | | Summary: This adds ability to mark test that do not complete due to hangs, crashes, etc., as "expected", to avoid flagging the build red for a known problem. Functionally, this extends the scope of the existing expectedFailureXXX decorators to cover these states as well. Once this is in, I will start replacing the magic list of failing tests in dosep.py with our regular annotations which should hopefully make code simpler. Reviewers: tfiala Subscribers: lldb-commits Differential Revision: http://reviews.llvm.org/D15530 llvm-svn: 255763
* Make few adjustments after r255542.Siva Chandra2015-12-151-3/+3
| | | | | | | | | | Reviewers: zturner Subscribers: lldb-commits Differential Revision: http://reviews.llvm.org/D15511 llvm-svn: 255584
* Remove the multiplier loop.Zachary Turner2015-12-141-7/+27
| | | | | | | | This is leading to some kind of subtle issue related to local functions and closures, so let's just go back to the old way for now. llvm-svn: 255567
* Make skipIf decorator support not_in() functor.Zachary Turner2015-12-141-6/+8
| | | | llvm-svn: 255542
* Make debug info specification use categories system.Zachary Turner2015-12-141-68/+20
| | | | | | | Reviewed By: Tamas Berghammer, Pavel Labath Differential Revision: http://reviews.llvm.org/D15428 llvm-svn: 255525
* Make test categories composablePavel Labath2015-12-141-2/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Previously the add_test_categories would simply overwrite the current set of categories for a method. This change makes the decorator truly "add" categories, by extending the current set of categories instead of replacing it. To do this, I have: - replaced the getCategories() property on a method (which was itself a method), with a simple list property "categories". This makes add_test_categories easier to implement, and test categories isn't something which should change between calls anyway. - rewritten the getCategoriesForTest function to merge method categories with the categories of the test case. Previously, it would just use the method categories if they were present. I have also greatly simplified this method. Originally, it would use a lot of introspection to enable it being called on various types of objects. Based on my tests, it was only ever being called on a test case. The new function uses much less introspection then the preivous one, so we should easily catch any stray uses, if there are any, as they will generate exceptions now. Reviewers: zturner, tfiala, tberghammer Subscribers: lldb-commits Differential Revision: http://reviews.llvm.org/D15451 llvm-svn: 255493
* Remove -S option from dotest.py.Zachary Turner2015-12-111-13/+1
| | | | llvm-svn: 255361
* Add test event marking a test as explicitly eligible for rerun if it is ↵Todd Fiala2015-12-111-0/+8
| | | | | | | | | | | | | | marked flakey. This will be used in a future change to support rerunning flakey tests that hit a test result isue in a low-load, single worker test runner phase. This is implemented as an additive-style event rather than being evaluated and added to the start_test event because the decorator code only runs after the start_test event is created and sent. i.e. LLDBTestResult.startTest() runs before the test method decorators run. llvm-svn: 255351
* Add NetBSD support in the buildDriver and buildLibrary routinesKamil Rytarowski2015-12-101-2/+2
| | | | | | | | | | | | Summary: NetBSD is like FreeBSD and Linux in these routines. Reviewers: clay.chang, tfiala, emaste, joerg Subscribers: lldb-commits, emaste Differential Revision: http://reviews.llvm.org/D15374 llvm-svn: 255308
* Remove -w option from dotest.py.Zachary Turner2015-12-101-12/+0
| | | | llvm-svn: 255275
* Remove -k command line option from dotest.py.Zachary Turner2015-12-091-24/+0
| | | | | | This is part of an effort to remove unused command line options. llvm-svn: 255143
* XFAIL TestReturnValue for remote Windows->Android testsPavel Labath2015-12-091-1/+3
| | | | | | this also adds the ability to match the host platform to the expectedFailureAll decorator. llvm-svn: 255105
* Remove the -c option from dotest.py.Zachary Turner2015-12-081-63/+1
| | | | | | | | | | This seems to be a legacy relic from days gone by where the remote test suite runner operated completely differently than it does today. git blames and comments traced this functionality back to about 2012, and nobody seems to know anything about it now. llvm-svn: 255060
* Remove the -b option from dotest.pyZachary Turner2015-12-081-12/+0
| | | | | | | This removes the blacklist option as part of an effort to remove unused / unmaintained command line options from the test suite. llvm-svn: 255040
* Remove +b option from dotest.pyZachary Turner2015-12-081-15/+1
| | | | llvm-svn: 255037
* Modify "platform connect" to connect to processes as wellTamas Berghammer2015-12-081-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | The standard remote debugging workflow with gdb is to start the application on the remote host under gdbserver (e.g.: gdbserver :5039 a.out) and then connect to it with gdb. The same workflow is supported by debugserver/lldb-gdbserver with a very similar syntax but to access all features of lldb we need to be connected also to an lldb-platform instance running on the target. Before this change this had to be done manually with starting a separate lldb-platform on the target machine and then connecting to it with lldb before connecting to the process. This change modifies the behavior of "platform connect" with automatically connecting to the process instance if it was started by the remote platform. With this command replacing gdbserver in a gdb based worflow is usually as simple as replacing the command to execute gdbserver with executing lldb-platform. Differential revision: http://reviews.llvm.org/D14952 llvm-svn: 255016
* Get rid of global variables in dotest.pyZachary Turner2015-12-081-39/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | This moves all the global variables into a separate module called `configuration`. This has a number of advantages: 1. Configuration data is centrally maintained so it's easy to get a high level overview of what configuration data the test suite makes use of. 2. The method of sharing configuration data among different parts of the test suite becomes standardized. Previously we would put some things into the `lldb` module, some things into the `lldbtest_config` module, and some things would not get shared. Now everything is shared through one module and is available to the entire test suite. 3. It opens the door to moving some of the initialization code into the `configuration` module, simplifying the implementation of `dotest.py`. There are a few stragglers that didn't get converted over to using the `configuration` module in this patch, because it would have grown the size of the patch unnecessarily. This includes everything currently in the `lldbtest_config` module, as well as the `lldb.remote_platform` variable. We can address these in the future. llvm-svn: 254982
* Add initial NetBSD support in lldbsuite/test/lldbtest.pyKamil Rytarowski2015-12-071-5/+21
| | | | | | | | | | | | | | | | | | | | | | | | Summary: Add new functions: - expectedFailureNetBSD() - expectedFlakeyNetBSD() - skipIfNetBSD() Add new NetBSD entry in: - getPlatform() - getHostPlatform() Assume that libc++ is installed and use the GNU toolchain Reviewers: joerg, emaste, tfiala, clayborg Subscribers: lldb-commits Differential Revision: http://reviews.llvm.org/D15262 llvm-svn: 254948
* Define new builder_netbsdKamil Rytarowski2015-12-051-0/+2
| | | | | | | | | | | | Summary: This is used in tests. Reviewers: emaste, tfiala, clayborg Subscribers: zturner, lldb-commits, joerg Differential Revision: http://reviews.llvm.org/D15065 llvm-svn: 254853
* Revert "Make skipIf support the not_in function (second attempt)."Siva Chandra2015-11-201-15/+11
| | | | | | | | | | | | Summary: This reverts commit 70dca28976ee8137acce2cc203dd394f4d761276. Reviewers: amccarth, zturner Subscribers: lldb-commits Differential Revision: http://reviews.llvm.org/D14881 llvm-svn: 253704
* Make skipIf support the not_in function (second attempt).Adrian McCarthy2015-11-201-11/+15
| | | | llvm-svn: 253683
* Revert "Make skipIf support the not_in function."Siva Chandra2015-11-201-3/+3
| | | | | | | | | | | | | | Summary: This reverts commit 251965377bdfb6227eea42c12a792c059e4e8a4b as a test marked "skipIf(compiler='gcc')" runs when testing with GCC. Reviewers: amccarth Subscribers: lldb-commits Differential Revision: http://reviews.llvm.org/D14852 llvm-svn: 253631
* Make skipIf support the not_in function.Adrian McCarthy2015-11-201-3/+3
| | | | llvm-svn: 253623
* Cleanup work required to get the ASAN tests to run in the new test suite ↵Enrico Granata2015-11-191-2/+3
| | | | | | infrastructure; the tests are now xfailed on Darwin pending investigation llvm-svn: 253604
* Support unix-abstract-connect scheme as platform url in lldb testsuiteYing Chen2015-11-181-2/+5
| | | | | | | | | | Reviewers: ovyalov Subscribers: lldb-commits Differential Revision: http://reviews.llvm.org/D14765 llvm-svn: 253488
* Fix some issues with swig & string conversion.Zachary Turner2015-11-181-1/+1
| | | | | | | | | | | | | | | | | | | | This patch fixes two issues: 1) Popen needs to be used with universal_newlines=True by default. This elicits automatic decoding from bytes -> string in Py3, and has no negative effects in other Py versions. 2) The swig typemaps for converting between string and (char*, int) did not work correctly when the length of the string was 0, indicating an error. In this case we would try to construct a string from uninitialized data. 3) Ironically, the bug mentioned in #2 led to a test passing on Windows that was actually broken, because the test was written such that the assertion was never even getting checked, so it passed by default. So we additionally fix this test to also fail if the method errors. By fixing this test it's now broken on Windows, so we also xfail it. llvm-svn: 253487
* Python 3 - Skip a certain test for a particular (swig,python) combo.Zachary Turner2015-11-161-1/+8
| | | | | | | | | | | Current versions of SWIG have a bug with Python 3 that causes Python to assert when iterating over a generator. This patch skips the test for the right combination of Python version and SWIG version. I'm attempting to upstream a patch to SWIG to fix this in a subsequent as-of-yet unreleased version, but I don't know how long that will take. llvm-svn: 253273
* Re-Apply "Add a "not_in()" function you can apply to the list type arguments ↵Ying Chen2015-11-161-3/+18
| | | | | | | | | | | | | | | | | to expectedFailureAll ..." with fix Summary: - Re-Commit r253106 - Initialize self.debug_info in Base::setUp() - Fix argument order when calling check_list_or_lambda for compiler Reviewers: jingham Subscribers: lldb-commits Differential Revision: http://reviews.llvm.org/D14673 llvm-svn: 253272
* Add the ability to xfail or skip based on swig / python version.Zachary Turner2015-11-161-11/+54
| | | | llvm-svn: 253263
* Revert "Add a "not_in()" function you can apply to the list type arguments ↵Pavel Labath2015-11-161-20/+3
| | | | | | | | to expectedFailureAll to reverse" This reverts commit r253106. llvm-svn: 253197
* Add a "not_in()" function you can apply to the list type arguments to ↵Jim Ingham2015-11-141-3/+20
| | | | | | | | expectedFailureAll to reverse the sense of the test. llvm-svn: 253106
* Python 3 - Don't use unbuffered I/O in text mode.Zachary Turner2015-11-071-2/+2
| | | | | | | | This is unsupported in Python 3. This could also have been fixed by using "wb" instead of "w", but it doesn't seem like writing the session log absolutely *needs* to be unbuffered. llvm-svn: 252381
* Don't use module internal implementation details in our decorators.Zachary Turner2015-11-061-8/+4
| | | | | | | | | | | | | | | | | We tried implementing something akin to a conditionalExpectedFailure decorator for unittest2. We did this by making use of some implementation details of the unittest2 module. In an effort to make this work with unittest, this patch removes the reliance on the implementation details. I have a hard time wrapping my head around how this all works with the deeply nested decorators, but the spirit of the patch here is to do do the following: If the condition function is true, use the original unittest2.expectedFailure decorator. Otherwise don't use any decorator, just call the test function. Differential Revision: http://reviews.llvm.org/D14406 Reviewed By: tberghammer, labath llvm-svn: 252326
* Python 3 - Turn on absolute imports, and fix existing imports.Zachary Turner2015-11-051-11/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Absolute imports were introduced in Python 2.5 as a feature (e.g. from __future__ import absolute_import), and made default in Python 3. When absolute imports are enabled, the import system changes in a couple of ways: 1) The `import foo` syntax will *only* search sys.path. If `foo` isn't in sys.path, it won't be found. Period. Without absolute imports, the import system will also search the same directory that the importing file resides in, so that you can easily import from the same folder. 2) From inside a package, you can use a dot syntax to refer to higher levels of the current package. For example, if you are in the package lldbsuite.test.utility, then ..foo refers to lldbsuite.test.foo. You can use this notation with the `from X import Y` syntax to write intra-package references. For example, using the previous locationa s a starting point, writing `from ..support import seven` would import lldbsuite.support.seven Since this is now the default behavior in Python 3, this means that importing from the same directory with `import foo` *no longer works*. As a result, the only way to have portable code is to force absolute imports for all versions of Python. See PEP 0328 [https://www.python.org/dev/peps/pep-0328/] for more information about absolute and relative imports. Differential Revision: http://reviews.llvm.org/D14342 Reviewed By: Todd Fiala llvm-svn: 252191
* Teach LLDB how to directly launch processes on the iOS simulatorEnrico Granata2015-11-051-0/+4
| | | | | | | | This allows for command-line debugging of iOS simulator binaries (as long as UI is not required, or a full UI simulator has previously been otherwise launched), as well as execution of the LLDB test suite on the iOS simulator This is known to compile on OSX 10.11 GM - feedback from people on other platforms and/or older versions of OSX as to the buildability of this code is greatly appreciated llvm-svn: 252112
* Remove `use_lldb_suite` from the package, and don't import it anymore.Zachary Turner2015-11-031-1/+1
| | | | | | | | | | | | | | | | This module was originally intended to be imported by top-level scripts to be able to find the LLDB packages and third party libraries. Packages themselves shouldn't need to import it, because by the time it gets into the package, the top-level script should have already done this. Indeed, it was just adding the same values to sys.path multiple times, so this patch is essentially no functional change. To make sure it doesn't get re-introduced, we also delete the `use_lldb_suite` module from `lldbsuite/test`, although the original copy still remains in `lldb/test` llvm-svn: 251963
* Tighten up sys.path, and use absolute imports everywhere.Zachary Turner2015-11-031-1/+1
| | | | | | | | | | | | | | | | | | | | For convenience, we had added the folder that dotest.py was in to sys.path, so that we could easily write things like `import lldbutil` from anywhere and any test. This introduces a subtle problem when using Python's package system, because when unittest2 imports a particular test suite, the test suite is detached from the package. Thus, writing "import lldbutil" from dotest imports it as part of the package, and writing the same line from a test does a fresh import since the importing module was not part of the same package. The real way to fix this is to use absolute imports everywhere. Instead of writing "import lldbutil", we need to write "import lldbsuite.test.util". This patch fixes up that and all other similar cases, and additionally removes the script directory from sys.path to ensure that this can't happen again. llvm-svn: 251886
* Move lldb/test to lldb/packages/Python/lldbsuite/test.Zachary Turner2015-10-281-0/+2838
This is the conclusion of an effort to get LLDB's Python code structured into a bona-fide Python package. This has a number of benefits, but most notably the ability to more easily share Python code between different but related pieces of LLDB's Python infrastructure (for example, `scripts` can now share code with `test`). llvm-svn: 251532
OpenPOWER on IntegriCloud