buildroot/support/testing, branch 2017.08

support/testing: allow to use a multiplier for timeouts

2017-08-10T08:08:06+00:00

Add a parameter to run-tests to act as a multiplier for all timeouts of emulator. It can be used to avoid sporadic failures on slow host machines as well in elastic runners on the cloud. Cc: Arnout Vandecappelle Cc: Thomas Petazzoni Signed-off-by: Ricardo Martincoski [Arnout: rename multiplier to timeout_multiplier everywhere] Signed-off-by: Arnout Vandecappelle (Essensium/Mind)

support/testing: add runtime testing for read-only systemd

2017-08-02T19:00:44+00:00

We add the 3 following combinations: - basic systemd, read-only, network w/ ifupdown - basic systemd, read-only, network w/ networkd - full systemd, read-only, network w/ networkd The tests just verify what the /sbin/init binary is, and that we were able to grab an IP address. More tests can be added later, for example to check each systemd features (journal, tmpfiles...) Signed-off-by: "Yann E. MORIN" Cc: Thomas Petazzoni [Arnout: regenerate .gitlab-ci.yml] Signed-off-by: Arnout Vandecappelle (Essensium/Mind) Signed-off-by: Thomas Petazzoni

support/testing: add runtime testing for init systems

2017-08-02T15:42:49+00:00

The "builtin" kernel does not boot a systemd-based system, so we resort to building the same one as currently used by our qemu_arm_vexpress_defconfig. We test the 8 following combinations: - busybox, read-only, without network - busybox, read-only, with network - busybox, read-write, without network - busybox, read-write, with network - basic systemd, read-write, network w/ ifupdown - basic systemd, read-write, network w/ networkd - full systemd, read-write, network w/ networkd - no init system, read-only, without network The tests just verify what the /sbin/init binary is, and that we were able to grab an IP address. More tests can be added later, for example to check each systemd features (journal, tmpfiles...) Signed-off-by: "Yann E. MORIN" Cc: Thomas Petazzoni Cc: Arnout Vandecappelle [Arnout: update .gitlab-ci.yml] Signed-off-by: Arnout Vandecappelle (Essensium/Mind) Signed-off-by: Thomas Petazzoni

testing/infra/emulator: remove qemu warnings about audio

2017-07-31T19:26:06+00:00

The default audio backend for qemu is configured at compile time. It generates annoying warning messages to qemu's stderr when running our tests, like these: pulseaudio: set_sink_input_volume() failed pulseaudio: Reason: Invalid argument pulseaudio: set_sink_input_mute() failed pulseaudio: Reason: Invalid argument Explicitly set the audio backend to "none" at runtime to remove those messages from our logs. There is no command line argument for this, so use an environment variable when starting qemu. Signed-off-by: Ricardo Martincoski Signed-off-by: Thomas Petazzoni

support/testing: add test of BR2_CCACHE with an external toolchain

2017-07-24T16:35:15+00:00

We reuse TestExternalToolchainBuildrootuClibc and add ccache to its configuration. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) Reviewed-by: Ricardo Martincoski Tested-by: Ricardo Martincoski Signed-off-by: Thomas Petazzoni

testing/infra/basetest: move jlevel logic to constructor

2017-07-24T15:51:06+00:00

As suggested by Arnout in [1]. While at it, simplify the logic by always appending the BR2_JLEVEL and defaulting to 0 (the value copied from Config.in is used for 5 years now and is very unlikely to change). [1] http://patchwork.ozlabs.org/patch/790525/ Suggested-by: Arnout Vandecappelle Signed-off-by: Ricardo Martincoski Cc: Arnout Vandecappelle Tested-by: "Yann E. MORIN" Reviewed-by: "Yann E. MORIN" Signed-off-by: Thomas Petazzoni

support/testing: fix run-tests -j

2017-07-24T15:50:52+00:00

Since commit cf3cd4388a652c9af27ef1c35622e2d0a55b99a9 the -j option is silently ignored. The configuration lines are processed using '\n'.join(). This function adds intervening occurrences of the separator, but the resulting string does not end at a separator. >>> "n".join(["a","b"]) 'anb' It results in a defconfig that does not end in a newline. When BR2_JLEVEL is added by -j logic to the defconfig it ends up concatenated to the last line of the defconfig. BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_ARM=yBR2_JLEVEL=7 The resulting .config has the default BR2_JLEVEL=0. Instead of just workaround this problem by adding a newline before BR2_JLEVEL when -j is used, make the defconfig to end in a newline since it is a more future-proof solution. Signed-off-by: Ricardo Martincoski Cc: Arnout Vandecappelle Cc: Thomas Petazzoni Cc: Yann E. MORIN Tested-by: "Yann E. MORIN" Acked-by: "Yann E. MORIN" Signed-off-by: Thomas Petazzoni

testing/infra/builder: dump defconfig to log

2017-07-24T15:50:45+00:00

The defconfig is composed on-the-fly by test infra + tests. Dump it to the logfile before running 'make olddefconfig' so it can easily analysed when debugging. Signed-off-by: Ricardo Martincoski Cc: "Yann E. MORIN" Signed-off-by: Thomas Petazzoni

testing/tests/package: add basic unit test for IPython

2017-07-22T20:55:05+00:00

Signed-off-by: Andrey Smirnov [Thomas: update .gitlab-ci.yml.] Signed-off-by: Thomas Petazzoni

testing/tests/package/test_python: allow to change timeout

2017-07-22T20:54:37+00:00

Depending on Python implementation used for testing, time it takes to perform a given test can vary pretty significantly. To accout for that allow individual test functions to specify different timeout value. Signed-off-by: Andrey Smirnov Signed-off-by: Thomas Petazzoni