buildroot/support/testing/run-tests, branch 2019.02-op-build

testing/infra: Add BRConfigTest as superclass of BRTest

2019-02-06T09:40:15+00:00

The git tests don't need to do a full build, they only need to do a configure and download and/or legal-info. More tests of that type will be added in the future. Therefore, we want to have a test base class that doesn't automatically do a full build in the setUp(). Add this new class as a superclass of the existing BRTest class, so we don't need to update existing tests. Only the code in run-tests that iterates over all subclasses of BRTest has to be adapted to use BRConfigTest instead. Signed-off-by: Ricardo Martincoski Cc: Arnout Vandecappelle Signed-off-by: Matthew Weber Signed-off-by: Arnout Vandecappelle (Essensium/Mind)

support/testing: fix python syntax

2018-06-10T13:56:25+00:00

Fix three issues with code style in our test infra: - 'print' is now a function, - exceptions need to be caught-assigned with the 'as' keyword, - old-style "%s"%() formatting is deprecated. Signed-off-by: "Yann E. MORIN" Cc: Ricardo Martincoski [Thomas: drop indices in format strings.] Signed-off-by: Thomas Petazzoni

testing/infra/builder: call make with empty env

2018-04-01T17:59:36+00:00

support/run-tests: export download dir

2017-10-10T21:33:07+00:00

support/testing: fix code style

2017-10-06T16:56:49+00:00

Fix the trivial warnings from flake8: - remove modules imported but unused; - use 2 lines before class or module level method; - remove blank line at end of file. Signed-off-by: Ricardo Martincoski Acked-by: "Yann E. MORIN" Signed-off-by: Arnout Vandecappelle (Essensium/Mind)

support/tests: use relative test_dir

2017-09-05T19:09:06+00:00

When we list the available tests, we use test_dir, which is set from the path of the script. However, when we run the tests, we use the hard-coded path. Ditto to find the config file. For consistency, always use test_dir. Signed-off-by: "Yann E. MORIN" Cc: Thomas Petazzoni Cc: Ricardo Martincoski Signed-off-by: Thomas Petazzoni

support/testing: allow to use a multiplier for timeouts

2017-08-10T08:08:06+00:00

Add a parameter to run-tests to act as a multiplier for all timeouts of emulator. It can be used to avoid sporadic failures on slow host machines as well in elastic runners on the cloud. Cc: Arnout Vandecappelle Cc: Thomas Petazzoni Signed-off-by: Ricardo Martincoski [Arnout: rename multiplier to timeout_multiplier everywhere] Signed-off-by: Arnout Vandecappelle (Essensium/Mind)

support/testing: run testcases in parallel

2017-07-01T14:05:14+00:00

Let the user to pass -t to set the number of testcases to run simultaneously. When -j is not specified, calculate it to split the available cores between the simultaneous testcases. Example of auto calculated -j for cpu_count 8: -t -j total 1 9 9 2 4 8 3 3 9 4 2 8 >=5 1 t Signed-off-by: Ricardo Martincoski Signed-off-by: Thomas Petazzoni

support/testing: allow to set BR2_JLEVEL

2017-07-01T14:05:14+00:00

Let the user to override the default BR2_JLEVEL used for each testcase. Signed-off-by: Ricardo Martincoski Signed-off-by: Thomas Petazzoni

support/testing/run-tests: help: put the one-letter form before the long form

2017-05-17T19:56:49+00:00

This is what the manpages usually do, and what Python does with the automatically-added -h/--help parameter: Before the change: $ ./support/testing/run-tests [...] optional arguments: -h, --help show this help message and exit --list, -l list of available test cases --all, -a execute all test cases After the change: $ ./support/testing/run-tests [...] optional arguments: -h, --help show this help message and exit -l, --list list of available test cases -a, --all execute all test cases Signed-off-by: Luca Ceresoli Signed-off-by: Thomas Petazzoni