buildroot/support/testing/infra, branch 2017.08.1

support/testing: allow to use a multiplier for timeouts

2017-08-10T08:08:06+00:00

Add a parameter to run-tests to act as a multiplier for all timeouts of emulator. It can be used to avoid sporadic failures on slow host machines as well in elastic runners on the cloud. Cc: Arnout Vandecappelle Cc: Thomas Petazzoni Signed-off-by: Ricardo Martincoski [Arnout: rename multiplier to timeout_multiplier everywhere] Signed-off-by: Arnout Vandecappelle (Essensium/Mind)

testing/infra/emulator: remove qemu warnings about audio

2017-07-31T19:26:06+00:00

The default audio backend for qemu is configured at compile time. It generates annoying warning messages to qemu's stderr when running our tests, like these: pulseaudio: set_sink_input_volume() failed pulseaudio: Reason: Invalid argument pulseaudio: set_sink_input_mute() failed pulseaudio: Reason: Invalid argument Explicitly set the audio backend to "none" at runtime to remove those messages from our logs. There is no command line argument for this, so use an environment variable when starting qemu. Signed-off-by: Ricardo Martincoski Signed-off-by: Thomas Petazzoni

testing/infra/basetest: move jlevel logic to constructor

2017-07-24T15:51:06+00:00

As suggested by Arnout in [1]. While at it, simplify the logic by always appending the BR2_JLEVEL and defaulting to 0 (the value copied from Config.in is used for 5 years now and is very unlikely to change). [1] http://patchwork.ozlabs.org/patch/790525/ Suggested-by: Arnout Vandecappelle Signed-off-by: Ricardo Martincoski Cc: Arnout Vandecappelle Tested-by: "Yann E. MORIN" Reviewed-by: "Yann E. MORIN" Signed-off-by: Thomas Petazzoni

support/testing: fix run-tests -j

2017-07-24T15:50:52+00:00

Since commit cf3cd4388a652c9af27ef1c35622e2d0a55b99a9 the -j option is silently ignored. The configuration lines are processed using '\n'.join(). This function adds intervening occurrences of the separator, but the resulting string does not end at a separator. >>> "n".join(["a","b"]) 'anb' It results in a defconfig that does not end in a newline. When BR2_JLEVEL is added by -j logic to the defconfig it ends up concatenated to the last line of the defconfig. BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_ARM=yBR2_JLEVEL=7 The resulting .config has the default BR2_JLEVEL=0. Instead of just workaround this problem by adding a newline before BR2_JLEVEL when -j is used, make the defconfig to end in a newline since it is a more future-proof solution. Signed-off-by: Ricardo Martincoski Cc: Arnout Vandecappelle Cc: Thomas Petazzoni Cc: Yann E. MORIN Tested-by: "Yann E. MORIN" Acked-by: "Yann E. MORIN" Signed-off-by: Thomas Petazzoni

testing/infra/builder: dump defconfig to log

2017-07-24T15:50:45+00:00

The defconfig is composed on-the-fly by test infra + tests. Dump it to the logfile before running 'make olddefconfig' so it can easily analysed when debugging. Signed-off-by: Ricardo Martincoski Cc: "Yann E. MORIN" Signed-off-by: Thomas Petazzoni

testing/infra/emulator: allow to specify pexpect timeout

2017-07-22T20:37:06+00:00

Some commands take more than 5 seconds to complete under QEMU, so add provisions to allow individual unit-test to specify different duration to avoid false negative test failures. Signed-off-by: Andrey Smirnov Reviewed-by: Ricardo Martincoski Tested-by: Ricardo Martincoski Signed-off-by: Thomas Petazzoni

support/tests: allow properly indented config fragment

2017-07-18T21:46:09+00:00

Currently, defining a config fragment in the runtime test infra requires that the fragment not to be indented. This is beark, and causes grievance when looking at the code (e.g. to fix it). Just strip out all leading spaces/tabs when writing the configuration lines into the config file, allowing in-line indented config fragments, like so: class TestFoo(bla): config = bla.config + \ """ FOO=y # BAR is not set """ Signed-off-by: "Yann E. MORIN" Cc: Thomas Petazzoni Cc: Arnout Vandecappelle Signed-off-by: Thomas Petazzoni

support/testing: unbreak run-tests -l

2017-07-10T21:51:33+00:00

In commit b78b50465c20c1733753a8dd47945cf80c9155f8, the initialisation of BRTest.builddir was moved to the __init__ function. However, it is set based on BRTest.outputdir and that is only set when the -o argument is given to run-tests. When called as "run-tests -l", there is no -o argument so BRTest.outputdir remains unset. To fix, keep BRTest.builddir at None when BRTest.outputdir is None. While we're at it, drop the direct access to the class member. If a subclass wishes to set outputdir to something else before calling BRTest.__init__, they are free to do so. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) Reported-by: "Yann E. MORIN" Tested-by: "Yann E. MORIN" Signed-off-by: Thomas Petazzoni

support/testing: move BRTest initialisation to init

2017-07-10T15:46:25+00:00

BRTest's setUp() method contains a few assignments that initialize its member variables. Since we will want to use these in test case overrides, move them to the __init__ function. Also allow the config member to be overridden, rather than always taking the class member. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) Signed-off-by: Thomas Petazzoni

support/testing: strip /usr/ part from HOST_DIR

2017-07-10T15:46:22+00:00

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) Signed-off-by: Thomas Petazzoni