buildroot/support/testing/infra/basetest.py, branch 2018.02

support/testing: use more recent toolchains

2017-10-21T17:48:30+00:00

With the hard disk crash of autobuild.b.o, we lost old toolchains, so use the latest toolchains, which have been restored. Signed-off-by: Thomas Petazzoni

support/testing: fix code style

2017-10-06T16:56:49+00:00

Fix the trivial warnings from flake8: - remove modules imported but unused; - use 2 lines before class or module level method; - remove blank line at end of file. Signed-off-by: Ricardo Martincoski Acked-by: "Yann E. MORIN" Signed-off-by: Arnout Vandecappelle (Essensium/Mind)

support/testing: standardize defconfig fragments style

2017-10-06T16:56:06+00:00

Change all defconfig fragments to take advantage of "cf3cd4388a support/tests: allow properly indented config fragment". Make each defconfig fragment: - start after a backslash; - be declared as a multi-line string literal; - be indented one level more than the variable that contains it. Signed-off-by: Ricardo Martincoski Acked-by: "Yann E. MORIN" Signed-off-by: Arnout Vandecappelle (Essensium/Mind)

support/testing: allow to indent ccache defconfig fragment

2017-10-06T16:53:13+00:00

Move the strip out of leading spaces in defconfig fragments from the BRTest class to the Builder class. It actually postpones the strip out, consequentially allowing test cases to post-process the defconfig in their own __init__ before calling the __init__ method from BRTest. Moving this code to the Builder class also allows any new test class that inherits from BRTest to reuse the same code even if the new class overrides the setUp method. At same time, prepend a newline to the jlevel handling otherwise it would stop working for defconfig fragments that don't end in a newline. Signed-off-by: Ricardo Martincoski Cc: Yann E. MORIN Signed-off-by: Arnout Vandecappelle (Essensium/Mind)

support/testing: allow to use a multiplier for timeouts

2017-08-10T08:08:06+00:00

Add a parameter to run-tests to act as a multiplier for all timeouts of emulator. It can be used to avoid sporadic failures on slow host machines as well in elastic runners on the cloud. Cc: Arnout Vandecappelle Cc: Thomas Petazzoni Signed-off-by: Ricardo Martincoski [Arnout: rename multiplier to timeout_multiplier everywhere] Signed-off-by: Arnout Vandecappelle (Essensium/Mind)

testing/infra/basetest: move jlevel logic to constructor

2017-07-24T15:51:06+00:00

As suggested by Arnout in [1]. While at it, simplify the logic by always appending the BR2_JLEVEL and defaulting to 0 (the value copied from Config.in is used for 5 years now and is very unlikely to change). [1] http://patchwork.ozlabs.org/patch/790525/ Suggested-by: Arnout Vandecappelle Signed-off-by: Ricardo Martincoski Cc: Arnout Vandecappelle Tested-by: "Yann E. MORIN" Reviewed-by: "Yann E. MORIN" Signed-off-by: Thomas Petazzoni

support/testing: fix run-tests -j

2017-07-24T15:50:52+00:00

Since commit cf3cd4388a652c9af27ef1c35622e2d0a55b99a9 the -j option is silently ignored. The configuration lines are processed using '\n'.join(). This function adds intervening occurrences of the separator, but the resulting string does not end at a separator. >>> "n".join(["a","b"]) 'anb' It results in a defconfig that does not end in a newline. When BR2_JLEVEL is added by -j logic to the defconfig it ends up concatenated to the last line of the defconfig. BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_ARM=yBR2_JLEVEL=7 The resulting .config has the default BR2_JLEVEL=0. Instead of just workaround this problem by adding a newline before BR2_JLEVEL when -j is used, make the defconfig to end in a newline since it is a more future-proof solution. Signed-off-by: Ricardo Martincoski Cc: Arnout Vandecappelle Cc: Thomas Petazzoni Cc: Yann E. MORIN Tested-by: "Yann E. MORIN" Acked-by: "Yann E. MORIN" Signed-off-by: Thomas Petazzoni

support/tests: allow properly indented config fragment

2017-07-18T21:46:09+00:00

Currently, defining a config fragment in the runtime test infra requires that the fragment not to be indented. This is beark, and causes grievance when looking at the code (e.g. to fix it). Just strip out all leading spaces/tabs when writing the configuration lines into the config file, allowing in-line indented config fragments, like so: class TestFoo(bla): config = bla.config + \ """ FOO=y # BAR is not set """ Signed-off-by: "Yann E. MORIN" Cc: Thomas Petazzoni Cc: Arnout Vandecappelle Signed-off-by: Thomas Petazzoni

support/testing: unbreak run-tests -l

2017-07-10T21:51:33+00:00

In commit b78b50465c20c1733753a8dd47945cf80c9155f8, the initialisation of BRTest.builddir was moved to the __init__ function. However, it is set based on BRTest.outputdir and that is only set when the -o argument is given to run-tests. When called as "run-tests -l", there is no -o argument so BRTest.outputdir remains unset. To fix, keep BRTest.builddir at None when BRTest.outputdir is None. While we're at it, drop the direct access to the class member. If a subclass wishes to set outputdir to something else before calling BRTest.__init__, they are free to do so. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) Reported-by: "Yann E. MORIN" Tested-by: "Yann E. MORIN" Signed-off-by: Thomas Petazzoni

support/testing: move BRTest initialisation to init

2017-07-10T15:46:25+00:00

BRTest's setUp() method contains a few assignments that initialize its member variables. Since we will want to use these in test case overrides, move them to the __init__ function. Also allow the config member to be overridden, rather than always taking the class member. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) Signed-off-by: Thomas Petazzoni