summaryrefslogtreecommitdiffstats
path: root/lldb/packages/Python/lldbsuite/test/issue_verification
Commit message (Collapse)AuthorAgeFilesLines
* fix a race is the LLDB test suite results collectionTodd Fiala2016-04-181-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | The race boiled down to this: If a test worker queue is able to run the test inferior and clean up before the dosep.py listener socket is spun up, and the worker queue is the last one (as would be the case when there's only one test rerunning in the rerun queue), then the test suite will exit the main loop before having a chance to process any test events coming from the test inferior or the worker queue job control. I found this race to be far more likely on fast hardware. Our Linux CI is one such example. While it will show up primarily during meta test events generated by a worker thread when a test inferior times out or exits with an exceptional exit (e.g. seg fault), it only requires that the OS takes longer to hook up the listener socket than it takes for the final test inferior and worker thread to shut down. See: http://reviews.llvm.org/D19214 reviewed by: Pavel Labath llvm-svn: 266624
* test infra: fix lldbinline tests to work with rerunTodd Fiala2015-12-223-0/+31
| | | | | | | Fixes: https://llvm.org/bugs/show_bug.cgi?id=25922 llvm-svn: 256255
* [test] Add ability to expect timeoutsPavel Labath2015-12-161-0/+1
| | | | | | | | | | | | | | | | | Summary: This adds ability to mark test that do not complete due to hangs, crashes, etc., as "expected", to avoid flagging the build red for a known problem. Functionally, this extends the scope of the existing expectedFailureXXX decorators to cover these states as well. Once this is in, I will start replacing the magic list of failing tests in dosep.py with our regular annotations which should hopefully make code simpler. Reviewers: tfiala Subscribers: lldb-commits Differential Revision: http://reviews.llvm.org/D15530 llvm-svn: 255763
* test infra: enable single-worker rerun phase for flakey tests.Todd Fiala2015-12-143-0/+73
| | | | | | | | | | | | | | | | Use of --rerun-all-issues will enable any test method failure, not just test methods marked with the flakey decorator, to rerun. Currently this does not change the flakey logic's immediate rerun attempt. I want to make sure this doesn't cause any significant issues before changing that part. The rerun reporting is only known to work properly with the default (new) BasicResultsFormatter reporting. Once we work out any issues, I'll go back and make sure the curses output handles it properly as well. llvm-svn: 255543
* test infra: adds book-keeping for rerunnable testsTodd Fiala2015-12-121-0/+24
| | | | | | | | | | | | | | Also adds full path info for exceptional exits and timeouts when no test method is currently running. Adds --rerun-all-issues command line arg. If specified, all test issues are eligible for rerun. If not specified, only tests marked flakey are eligible for rerun. The actual rerunning will occur in an upcoming change. This change just handles tha accounting of what should be rerun. llvm-svn: 255438
* Add expected timeout support to test event architecture.Todd Fiala2015-12-111-0/+19
| | | | llvm-svn: 255363
* enable timeout/exceptional exit support for xUnit formatterTodd Fiala2015-12-092-0/+40
| | | | | | | Also adds enable.py/disable.py script to simplify turning on and off the issue_verification tests helpful for testing a results formatter. llvm-svn: 255161
* Disable the issue verification tests.Todd Fiala2015-12-093-0/+0
| | | | llvm-svn: 255134
* create 3 issues for testbots: FAIL, ERROR (exceptional), and TIMEOUTTodd Fiala2015-12-094-0/+66
This change is a trial balloon to verify that the default test summary output sends the right output for the buildbot issue detection script. The effect of this change will be reverted after verifying the testbot behavior. This change will not stay in as is and will knowingly create noise, see this thread: http://lists.llvm.org/pipermail/lldb-dev/2015-December/009048.html llvm-svn: 255131
OpenPOWER on IntegriCloud