From 7574241053ef6407a75043cef6418272242bd5e0 Mon Sep 17 00:00:00 2001 From: Dan Liew Date: Sun, 27 Dec 2015 14:03:49 +0000 Subject: [lit] Implement support of per test timeout in lit. This should work with ShTest (executed externally or internally) and GTest test formats. To set the timeout a new option ``--timeout=`` has been added which specifies the maximum run time of an individual test in seconds. By default this 0 which causes no timeout to be enforced. The timeout can also be set from a lit configuration file by modifying the ``lit_config.maxIndividualTestTime`` property. To implement a timeout we now require the psutil Python module if a timeout is requested. This dependency is confined to the newly added ``lit.util.killProcessAndChildren()``. A note has been added into the TODO document describing how we can remove the dependency on the ``pustil`` module in the future. It would be nice to remove this immediately but that is a lot more work and Daniel Dunbar believes it is better that we get a working implementation first and then improve it. To avoid breaking the existing behaviour the psutil module will not be imported if no timeout is requested. The included testcases are derived from test cases provided by Jonathan Roelofs which were in an previous attempt to add a per test timeout to lit (http://reviews.llvm.org/D6584). Thanks Jonathan! Reviewers: ddunbar, jroelofs, cmatthews, MatzeB Subscribers: cmatthews, llvm-commits Differential Revision: http://reviews.llvm.org/D14706 llvm-svn: 256471 --- llvm/utils/lit/tests/googletest-timeout.py | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) create mode 100644 llvm/utils/lit/tests/googletest-timeout.py (limited to 'llvm/utils/lit/tests/googletest-timeout.py') diff --git a/llvm/utils/lit/tests/googletest-timeout.py b/llvm/utils/lit/tests/googletest-timeout.py new file mode 100644 index 00000000000..46acf32b3a6 --- /dev/null +++ b/llvm/utils/lit/tests/googletest-timeout.py @@ -0,0 +1,29 @@ +# REQUIRES: python-psutil + +# Check that the per test timeout is enforced when running GTest tests. +# +# RUN: not %{lit} -j 1 -v %{inputs}/googletest-timeout --timeout=1 > %t.cmd.out +# RUN: FileCheck < %t.cmd.out %s + +# Check that the per test timeout is enforced when running GTest tests via +# the configuration file +# +# RUN: not %{lit} -j 1 -v %{inputs}/googletest-timeout \ +# RUN: --param set_timeout=1 > %t.cfgset.out 2> %t.cfgset.err +# RUN: FileCheck < %t.cfgset.out %s + +# CHECK: -- Testing: +# CHECK: PASS: googletest-timeout :: DummySubDir/OneTest/FirstTest.subTestA +# CHECK: TIMEOUT: googletest-timeout :: DummySubDir/OneTest/FirstTest.subTestB +# CHECK: TIMEOUT: googletest-timeout :: DummySubDir/OneTest/FirstTest.subTestC +# CHECK: Expected Passes : 1 +# CHECK: Individual Timeouts: 2 + +# Test per test timeout via a config file and on the command line. +# The value set on the command line should override the config file. +# RUN: not %{lit} -j 1 -v %{inputs}/googletest-timeout \ +# RUN: --param set_timeout=1 --timeout=2 > %t.cmdover.out 2> %t.cmdover.err +# RUN: FileCheck < %t.cmdover.out %s +# RUN: FileCheck --check-prefix=CHECK-CMDLINE-OVERRIDE-ERR < %t.cmdover.err %s + +# CHECK-CMDLINE-OVERRIDE-ERR: Forcing timeout to be 2 seconds -- cgit v1.2.3