summaryrefslogtreecommitdiffstats
path: root/support/testing/tests
Commit message (Collapse)AuthorAgeFilesLines
* tests: add test for post-fakeroot scriptYann E. MORIN2018-01-154-13/+7
| | | | | | | | | | | | | | | | | | Currently, only post-build and post-image scripts were tested, each with their own test-script. The two test-scripts only differ in the name of the log file they create, and it is based on the name of the script, so it is easy to share the script. This allows us to easily re-use it for testing post-fakeroot scripts. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com> Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* core/tests: add test for compressed iso9660Yann E. MORIN2017-12-311-0/+41
| | | | | | | Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> [Thomas: really set the transparent compression option, take into account the renaming of the option.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/testing: TestATFMarvell needs BR2_TARGET_UBOOT_NEEDS_DTC=yThomas Petazzoni2017-12-121-0/+1
| | | | | | | | | | | | The Marvell U-Boot needs DTC to build, so the test case fails to build on systems where dtc isn't available. We add BR2_TARGET_UBOOT_NEEDS_DTC=y to make sure that we build our own DTC. Fixes: https://gitlab.com/buildroot.org/buildroot/-/jobs/44126707 Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/testing: add tests for ATFThomas Petazzoni2017-12-082-0/+79
| | | | | | | | | These new tests only do build tests, but allow to quickly verify that the ATF/U-Boot combinations for vexpress, Allwinner and Marvell platforms all continue to build properly. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* support/testing: update ISO9660 test case Linux kernelThomas Petazzoni2017-10-211-1/+1
| | | | | | | | | | | The Linux 4.0 kernel doesn't build with gcc 6.x, which is used since the toolchain update in commit 193dfffa834a4cd76bc7b41089bd93d4c37dfc65 ("support/testing: use more recent toolchains"). So let's update to Linux 4.11 instead (like the existing Qemu x86 defconfig does), and update the kernel configuration file accordingly. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/testing: use more recent toolchainsThomas Petazzoni2017-10-212-6/+5
| | | | | | | With the hard disk crash of autobuild.b.o, we lost old toolchains, so use the latest toolchains, which have been restored. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/testing: fix remaining code styleRicardo Martincoski2017-10-061-2/+2
| | | | | | | | | | Fix the remaining code style warnings from flake8: - properly indent continuation lines; - use proper code to test a parameter is not None. Cc: Yann E. MORIN <yann.morin.1998@free.fr> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* testing/tests/init: use lowercase method namesRicardo Martincoski2017-10-064-39/+39
| | | | | | | | | | | | | | Use method naming convention from PEP8 as other test cases already do. sed \ -e 's,startEmulator,start_emulator,g' \ -e 's,checkInit,check_init,g' \ -e 's,checkNetwork,check_network,g' \ -i support/testing/tests/init/*.py Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* support/testing: fix code styleRicardo Martincoski2017-10-0617-19/+48
| | | | | | | | | | | Fix the trivial warnings from flake8: - remove modules imported but unused; - use 2 lines before class or module level method; - remove blank line at end of file. Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* support/testing: standardize defconfig fragments styleRicardo Martincoski2017-10-0613-231/+236
| | | | | | | | | | | | | | Change all defconfig fragments to take advantage of "cf3cd4388a support/tests: allow properly indented config fragment". Make each defconfig fragment: - start after a backslash; - be declared as a multi-line string literal; - be indented one level more than the variable that contains it. Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* boot/grub: removeYann E. MORIN2017-09-231-37/+0
| | | | | | | | | | | | | | | | | | | | | | grub is no longer maintained: it is stuck at version 0.97 with huge patches that have no opportunity to be applied upstream, as upstream has even renamed it grub-legacy. Besides, it no longer builds correctly with recent binutils versions, and even the huge patches we could grab from Debian do not help the slightest. Since upstream really considers it dead, and there are at least two alternatives (grub2 and syslinux), just remove grub. Add a legacy entry. Remove the test cases as well. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* support/testing: add runtime testing for read-only systemdYann E. MORIN2017-08-022-0/+71
| | | | | | | | | | | | | | | | | | We add the 3 following combinations: - basic systemd, read-only, network w/ ifupdown - basic systemd, read-only, network w/ networkd - full systemd, read-only, network w/ networkd The tests just verify what the /sbin/init binary is, and that we were able to grab an IP address. More tests can be added later, for example to check each systemd features (journal, tmpfiles...) Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> [Arnout: regenerate .gitlab-ci.yml] Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/testing: add runtime testing for init systemsYann E. MORIN2017-08-025-0/+228
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The "builtin" kernel does not boot a systemd-based system, so we resort to building the same one as currently used by our qemu_arm_vexpress_defconfig. We test the 8 following combinations: - busybox, read-only, without network - busybox, read-only, with network - busybox, read-write, without network - busybox, read-write, with network - basic systemd, read-write, network w/ ifupdown - basic systemd, read-write, network w/ networkd - full systemd, read-write, network w/ networkd - no init system, read-only, without network The tests just verify what the /sbin/init binary is, and that we were able to grab an IP address. More tests can be added later, for example to check each systemd features (journal, tmpfiles...) Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> [Arnout: update .gitlab-ci.yml] Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/testing: add test of BR2_CCACHE with an external toolchainArnout Vandecappelle2017-07-241-0/+11
| | | | | | | | | | We reuse TestExternalToolchainBuildrootuClibc and add ccache to its configuration. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* testing/tests/package: add basic unit test for IPythonAndrey Smirnov2017-07-221-0/+38
| | | | | | Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com> [Thomas: update .gitlab-ci.yml.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* testing/tests/package/test_python: allow to change timeoutAndrey Smirnov2017-07-221-8/+8
| | | | | | | | | Depending on Python implementation used for testing, time it takes to perform a given test can vary pretty significantly. To accout for that allow individual test functions to specify different timeout value. Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* testing/tests/package/test_python: do not hardcode interpreter nameAndrey Smirnov2017-07-221-4/+6
| | | | | | | | | | In order to be able to leverage the same test code for testing different python interpreters (or wrappers around CPython) allow child classes of TestPythonBase to override the name of the executable used to run tests. Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* testing/tests/package/test_python: add TestPython3Andrey Smirnov2017-07-221-2/+15
| | | | | | | | | | | | Add Python3 version of TestPython2 to make sure both versions of Python get unit-tested. Modify the code of libc_time_test() to support that change (convert the code to use Python3-style "print"). Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com> [Thomas: update .gitlab-ci.yml.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* testing/tests/package/test_python: refactor TestPythonBaseAndrey Smirnov2017-07-221-4/+19
| | | | | | | | | | | | | | | | Convert TestPythonBase to a true base class that only provides code implementing various tests without defining tests themselves in a "discoverable" form. To retain correct testing functionality, add TestPython2 derived class that uses code from TestPythonBase to define actual runnable test. Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com> Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> [Thomas: fix typo in commit log, update .gitlab-ci.yml, both pointed by Ricardo.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/tests: fix ext4 runtime testYann E. MORIN2017-07-141-1/+1
| | | | | | | | | | | | The current test fails because of a legacy option, renamed during the recent ext overhaul. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Sébastien Szymanski <sebastien.szymanski@armadeus.com> Cc: Samuel Martin <s.martin49@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/testing: remove references to host/usrArnout Vandecappelle2017-07-053-3/+3
| | | | | | | The tools are now installed in host/bin instead of host/usr/bin. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/testing: add tests for musl and uclibc toolchainsThomas Petazzoni2017-07-051-0/+75
| | | | | | | | | | | | | | These tests simply build a system with musl and uclibc toolchains, and boot them under qemu. It allows to minimally validate that our support for musl/uclibc external toolchains is working. We already had some tests covering glibc toolchains, so we can now easily test that all three C libraries are supported. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> --- This commit is part of the series, as I've written/used those tests to validate that things are still working correctly with all of glibc, uclibc and musl toolchains.
* support/testing: rename variables for readabilityLuca Ceresoli2017-05-281-4/+4
| | | | | | | | | | | | | The 'lines' variable is overwritten with its own fields. Thus it contains a line first, and then a list of fields -- it never contains 'lines'. Use two different variables named 'line' and 'fields' to make the code more readable. Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net> Acked-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/testing: fs/ext: add a negative test for the "extent" featureLuca Ceresoli2017-05-281-0/+1
| | | | | | | | | We only have a positive test for it, in ext4. Let's have a negative one as well. Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net> Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/testing: rename check_broken_links to has_broken_linksLuca Ceresoli2017-05-171-3/+3
| | | | | | | | | has_broken_links makes it self-explanatory that this is a predicate function, and that the return value tells whether there _are_ broken links, not the opposite. Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/testing: add toolchain testsThomas Petazzoni2017-05-072-0/+156
| | | | | | | | This commit adds an initial toolchain test case, testing the ARM CodeSourcery toolchain, just checking that the proper sysroot is used, and that a minimal Linux system boots fine under Qemu. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/testing: add package testsThomas Petazzoni2017-05-073-0/+63
| | | | | | | | This commit adds some basic tests for two Buildroot packages: python and dropbear. These tests are by no mean meant to be exhaustive, but mainly to serve as initial examples for other tests. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/testing: add fs testsThomas Petazzoni2017-05-077-0/+414
| | | | | | | | | | This commit adds a number of test cases for various filesystem formats: ext2/3/4, iso9660, jffs2, squashfs, ubi/ubifs and yaffs2. All of them except yaffs2 are runtime tested. The iso9660 set of test cases is particularly rich, testing the proper operation of the iso9660 support with all of grub, grub2 and isolinux. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/testing: add core testsThomas Petazzoni2017-05-078-0/+159
| | | | | | | | | | | This commit adds a few Buildroot "core" tests, testing functionalities such as: - post-build and post-image scripts - root filesystem overlays - timezone support Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/testing: core testing infrastructureThomas Petazzoni2017-05-071-0/+0
This commit adds the core of a new testing infrastructure that allows to perform runtime testing of Buildroot generated systems. This infrastructure uses the Python unittest logic as its foundation. This core infrastructure commit includes the following aspects: - A base test class, called BRTest, defined in support/testing/infra/basetest.py. This base test class inherited from the Python provided unittest.TestCase, and must be subclassed by all Buildroot test cases. Its main purpose is to provide the Python unittest setUp() and tearDown() methods. In our case, setUp() takes care of building the Buildroot system described in the test case, and instantiate the Emulator object in case runtime testing is needed. The tearDown() method simply cleans things up (stop the emulator, remove the output directory). - A Builder class, defined in support/testing/infra/builder.py, simply responsible for building the Buildroot system in each test case. - An Emulator class, defined in support/testing/infra/emulator.py, responsible for running the generated system under Qemu, allowing each test case to run arbitrary commands inside the emulated system. - A run-tests script, which is the entry point to start the tests. Even though I wrote the original version of this small infrastructure, a huge amount of rework and improvement has been done by Maxime Hadjinlian, and squashed into this patch. So many thanks to Maxime for cleaning up and improving my Python code! Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
OpenPOWER on IntegriCloud