buildroot/support/testing/infra, branch 2017.11.2

support/testing: use more recent toolchains

2017-10-21T17:48:30+00:00

With the hard disk crash of autobuild.b.o, we lost old toolchains, so use the latest toolchains, which have been restored. Signed-off-by: Thomas Petazzoni

support/testing: fix code style

2017-10-06T16:56:49+00:00

Fix the trivial warnings from flake8: - remove modules imported but unused; - use 2 lines before class or module level method; - remove blank line at end of file. Signed-off-by: Ricardo Martincoski Acked-by: "Yann E. MORIN" Signed-off-by: Arnout Vandecappelle (Essensium/Mind)

support/testing: standardize defconfig fragments style

2017-10-06T16:56:06+00:00

Change all defconfig fragments to take advantage of "cf3cd4388a support/tests: allow properly indented config fragment". Make each defconfig fragment: - start after a backslash; - be declared as a multi-line string literal; - be indented one level more than the variable that contains it. Signed-off-by: Ricardo Martincoski Acked-by: "Yann E. MORIN" Signed-off-by: Arnout Vandecappelle (Essensium/Mind)

support/testing: allow to indent ccache defconfig fragment

2017-10-06T16:53:13+00:00

Move the strip out of leading spaces in defconfig fragments from the BRTest class to the Builder class. It actually postpones the strip out, consequentially allowing test cases to post-process the defconfig in their own __init__ before calling the __init__ method from BRTest. Moving this code to the Builder class also allows any new test class that inherits from BRTest to reuse the same code even if the new class overrides the setUp method. At same time, prepend a newline to the jlevel handling otherwise it would stop working for defconfig fragments that don't end in a newline. Signed-off-by: Ricardo Martincoski Cc: Yann E. MORIN Signed-off-by: Arnout Vandecappelle (Essensium/Mind)

support/testing: allow to use a multiplier for timeouts

2017-08-10T08:08:06+00:00

Add a parameter to run-tests to act as a multiplier for all timeouts of emulator. It can be used to avoid sporadic failures on slow host machines as well in elastic runners on the cloud. Cc: Arnout Vandecappelle Cc: Thomas Petazzoni Signed-off-by: Ricardo Martincoski [Arnout: rename multiplier to timeout_multiplier everywhere] Signed-off-by: Arnout Vandecappelle (Essensium/Mind)

testing/infra/emulator: remove qemu warnings about audio

2017-07-31T19:26:06+00:00

The default audio backend for qemu is configured at compile time. It generates annoying warning messages to qemu's stderr when running our tests, like these: pulseaudio: set_sink_input_volume() failed pulseaudio: Reason: Invalid argument pulseaudio: set_sink_input_mute() failed pulseaudio: Reason: Invalid argument Explicitly set the audio backend to "none" at runtime to remove those messages from our logs. There is no command line argument for this, so use an environment variable when starting qemu. Signed-off-by: Ricardo Martincoski Signed-off-by: Thomas Petazzoni

testing/infra/basetest: move jlevel logic to constructor

2017-07-24T15:51:06+00:00

As suggested by Arnout in [1]. While at it, simplify the logic by always appending the BR2_JLEVEL and defaulting to 0 (the value copied from Config.in is used for 5 years now and is very unlikely to change). [1] http://patchwork.ozlabs.org/patch/790525/ Suggested-by: Arnout Vandecappelle Signed-off-by: Ricardo Martincoski Cc: Arnout Vandecappelle Tested-by: "Yann E. MORIN" Reviewed-by: "Yann E. MORIN" Signed-off-by: Thomas Petazzoni

support/testing: fix run-tests -j

2017-07-24T15:50:52+00:00

Since commit cf3cd4388a652c9af27ef1c35622e2d0a55b99a9 the -j option is silently ignored. The configuration lines are processed using '\n'.join(). This function adds intervening occurrences of the separator, but the resulting string does not end at a separator. >>> "n".join(["a","b"]) 'anb' It results in a defconfig that does not end in a newline. When BR2_JLEVEL is added by -j logic to the defconfig it ends up concatenated to the last line of the defconfig. BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_ARM=yBR2_JLEVEL=7 The resulting .config has the default BR2_JLEVEL=0. Instead of just workaround this problem by adding a newline before BR2_JLEVEL when -j is used, make the defconfig to end in a newline since it is a more future-proof solution. Signed-off-by: Ricardo Martincoski Cc: Arnout Vandecappelle Cc: Thomas Petazzoni Cc: Yann E. MORIN Tested-by: "Yann E. MORIN" Acked-by: "Yann E. MORIN" Signed-off-by: Thomas Petazzoni

testing/infra/builder: dump defconfig to log

2017-07-24T15:50:45+00:00

The defconfig is composed on-the-fly by test infra + tests. Dump it to the logfile before running 'make olddefconfig' so it can easily analysed when debugging. Signed-off-by: Ricardo Martincoski Cc: "Yann E. MORIN" Signed-off-by: Thomas Petazzoni

testing/infra/emulator: allow to specify pexpect timeout

2017-07-22T20:37:06+00:00

Some commands take more than 5 seconds to complete under QEMU, so add provisions to allow individual unit-test to specify different duration to avoid false negative test failures. Signed-off-by: Andrey Smirnov Reviewed-by: Ricardo Martincoski Tested-by: Ricardo Martincoski Signed-off-by: Thomas Petazzoni