summaryrefslogtreecommitdiffstats
path: root/utils/genrandconfig
Commit message (Collapse)AuthorAgeFilesLines
* utils/genrandconfig: drop qt4 fixupsPeter Korsgaard2019-02-061-8/+0
| | | | | | | Now that Qt4 is removed, these fixups can be removed as well. Signed-off-by: Peter Korsgaard <peter@korsgaard.com> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* utils/genrandconfig: fix flake8 warningsThomas Petazzoni2018-12-031-2/+2
| | | | | | | | | | | Fixes: utils/genrandconfig:369:17: E231 missing whitespace after ',' utils/genrandconfig:370:1: E101 indentation contains mixed spaces and tabs utils/genrandconfig:370:1: W191 indentation contains tabs utils/genrandconfig:372:1: E101 indentation contains mixed spaces and tabs Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* utils/genrandconfig: test with BR2_OPTIMIZE_2=yEvgeniy Didin2018-12-031-0/+2
| | | | | | | | | | | | | | | | | Currently all random defconfigs which are used in autobuilder use size optimizaion (-Os), since BR2_OPTIMIZE_S=y is the default. Adding "-O2" optimization will give better test coverage. In many cases software gets built with speed optimization rather than size optimization. So let's add Level 2 optimizaion option to be generated in random defconfigs, so we could be able to test how packages are built with "-O2" in autobuilder. Signed-off-by: Evgeniy Didin <Evgeniy.Didin@synopsys.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: arc-buildroot@synopsys.com Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* utils/genrandconfig: add missing new line when creating the configurationThomas Petazzoni2018-11-181-1/+1
| | | | | | | | | | | | | | | | | | When adding the custom BR2_WGET value in the configuration, genrandconfig forgets to add a newline. Due to this, the next option that is added is printed on the same line as BR2_WGET="", which causes it to be ignored. Due to this, in all builds, the line right after BR2_WGET was ignored. It could have been BR2_ENABLE_DEBUG, BR2_INIT_BUSYBOX, BR2_INIT_SYSTEMD, BR2_ROOTFS_DEVICE_CREATION_DYNAMIC_EUDEV, BR2_STATIC_LIBS or BR2_PACKAGE_PYTHON_PY_ONLY depending on the randomization. Fix that by adding a proper newline at the end of the BR2_WGET option. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* support/config-fragments/autobuild: add Linaro AArch64 BE supportOfer Heifetz2018-06-281-0/+1
| | | | | | Signed-off-by: Ofer Heifetz <oferh@marvell.com> Reviewed-by: Romain Naour <romain.naour@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* utils/genrandconfig: use --no-check-certificate in wget by defaultThomas Petazzoni2018-03-311-0/+3
| | | | | | | | | | | | | | | | A number of autobuilder failures are due to the fact that autobuilder instances use old distributions, with old SSL certificates, and therefore wget aborts with an error "The certificate of `xyz.org' is not trusted.". In order to avoid such failures that are not very interesting in the context of the autobuilders, we pass --no-check-certificate to wget. The integrity of the downloaded files is anyway verified by the hashes, and this is only meant to be used in the context of testing/CI, not in production. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* utils/genrandconfig: adjust to core-dependencies removalThomas Petazzoni2018-03-301-1/+1
| | | | | | | | | | | | | | | | | | | | In commit ea9669fffae7d807ccb3810eb6c662cbb9e8f96f ("core: kill DEPENDENCIES_HOST_PREREQ"), the core-dependencies make target was removed, and is now named just "dependencies". This broke the utils/genrandconfig script, and this commit intends to fix that. Since this script is part of the Buildroot tree, it is provided together with Buildroot, so we don't need to support the legacy core-dependencies target. Someone checking out an older Buildroot will have a working setup, with support/dependencies exposing core-dependencies and utils/genrandconfig using core-dependencies. The only broken situation will be between ea9669fffae7d807ccb3810eb6c662cbb9e8f96f and this commit, but that's not a sufficient reason to add some backward compatibility code. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* genrandconfig: fix code styleRicardo Martincoski2018-03-131-17/+19
| | | | | | | | | | | | | | Fix these warnings: E201 whitespace after '[' E202 whitespace before ']' E501 line too long (138 > 132 characters) -> isolate the common part of the external toolchain url in a variable to make the long lines shorter and more readable. Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* utils/genrandconfig: check core-dependenciesArnout Vandecappelle (Essensium/Mind)2018-02-081-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When generating a random configuration, also check the host dependencies for that configuration and error out if it fails. This is mainly intended for the autobuilders, so they don't try to build a configuration for which the necessary host packages are not installed. Instead of erroring out, we could also generate a different random config. However: - When called for the autobuilders, the autobuild-run script will anyway just attempt another build. The only difference is that a warning will be printed in this autobuild log now. This warning is in fact quite useful to hint that the autobuilder could benefit from intalling this additional host package. - When called manually, the user may in fact want to install the missing host package and continue with the saved random config. Fixes: http://autobuild.buildroot.net/results/d7c/d7c12e873a71fef1e0b5e19cfad45024050614e1 http://autobuild.buildroot.net/results/a0a/a0a105c1ec2d0517112b02932a85831d4b597b9f http://autobuild.buildroot.net/results/96c/96cc6cedc64c777e993671204809599a1498472a and many others... Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* utils/genrandconfig: filter empty lines and comments in CSV fileThomas Petazzoni2017-11-271-1/+3
| | | | | | | | | | In preparation for the addition of comments in the CSV file listing toolchain configurations, we filter out such lines when reading the CSV file in utils/genrandconfig. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* genrandconfig: get configs from in-tree toolchain-configs.csvArnout Vandecappelle2017-07-251-11/+14
| | | | | | | | | | | | | | | | | | | | Now we have the toolchain config fragments in the buildroot directory itself, it is no longer necessary to fetch it from the toolchain URL. The --toolchains-url option is renamed to --toolchains-csv. The paths in the toolchains_csv file should be either absolute, or relative to buildrootdir. After this change, the script should be called from autobuild-run as: subprocess.call([os.path.join(srcdir, "utils/genrandconfig"), "-o", outputdir, "-b", srcdir, "--toolchains-csv", kwargs['toolchains_csv']], stdout=devnull, stderr=log) Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* genrandconfig: use minimal.configArnout Vandecappelle2017-07-251-8/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This has a number of side-effects which must be handled. The lines in minimal.config may be overridden by the random lines added by amending the configuration, so is_toolchain_usable() shouldn't take those into account, or indeed the random lines added. Therefore, make a copy of the config before appending minimal.config and the random lines. While we're at it, rename the variable to the more appropriate toolchainconfig. minimal.config sets BR2_INIT_NONE=y, but we really also want to test with BR2_INIT_BUSYBOX=y. Therefore, add a random line to use the busybox init system. We set its probability rather high. The probabilities of systemd and eudev are increased since they're now in the else branch of BR2_INIT_BUSYBOX, which halves the probability that we even get there. We now also generate configurations without busybox. Previously, busybox was almost always selected due to BR2_INIT_BUSYBOX=y. Only if systemd is selected there was a chance to build without busybox. We now set BR2_SYSTEM_BIN_SH_NONE=y, the other /bin/sh options are never tested. However, this is not really something that is relevant to test in the autobuilders. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* genrandconfig: fix the case when outputdir is 'output'Arnout Vandecappelle2017-07-251-1/+4
| | | | | | | | | | | | | When outputdir is 'output' (the default), genrandconfig didn't work correctly because it expects the configfile in outputdir, while Buildroot puts it in the buildroot directory. Fix this by explicitly checking if outputdir == buildrootdir/output. Because abspath is used for both paths, string comparison works reliably. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* genrandconfig: calculate configfile only onceArnout Vandecappelle2017-07-251-8/+9
| | | | | | | | | | | | | The path to the .config file is calculated in several places - replace it with a single calculation, and pass configfile as an argument to is_toolchain_usable and fixup_config. These functions also don't need outputdir any more. This makes it easier to fix the case when configfile is not in outputdir. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* genrandconfig: pass outputdir and buildrootdir as argumentsArnout Vandecappelle2017-07-251-15/+10
| | | | | | | | | | | | | | | | | | | | | | | | | The --instance argument is just an artifact of genrandconfig's history as part of autobuild-run. It is much more logical to pass the output directory and the buildroot directory as arguments, with sane defaults. This also allows us to remove the hack of creating a symlink in the instance directory if it doesn't exist yet. Note that the default outputdir 'output' doesn't work yet, because in that case Buildroot will put the config file in the buildroot directory instead of the output directory. This will be fixed in a follow-up patch. After this change, the script should be called from autobuild-run as: subprocess.call([os.path.join(srcdir, "utils/genrandconfig"), "-o", outputdir, "-b", srcdir, "--toolchains-url", kwargs['toolchains_url']], stdout=devnull, stderr=log) Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* genrandconfig: calculate buildrootdir in __main__Arnout Vandecappelle2017-07-251-8/+6
| | | | | | | This prepares for passing buildrootdir as an argument. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* genrandconfig: calculate outputdir in __main__Arnout Vandecappelle2017-07-251-14/+14
| | | | | | | This prepares for passing outputdir as an argument. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* genrandconfig: verbose output and use stderrArnout Vandecappelle2017-07-251-49/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | The output of genrandconfig is currently very terse, which is annoying for debugging the script or generally seeing what is going on. Also the timing information added by log_write isn't very useful when the script is used stand-alone. In the new setup, (verbose) output goes to stdout and error output goes to stderr. Also the "INFO: generate the configuration" message is eliminated - it should go in the autobuild-run script. We also add an explicit message when a toolchain can't be used after the first defconfig, otherwise autobuild-run will just silently restart. Note that, since the output of make is no longer redirected to /dev/null, we get one more message on stderr that will be recorded in the autobuilder's log file: KCONFIG_SEED=0xXXXXXXXX. This approach allows us to optimise the error handling to use exceptions, where appropriate, which can be caught at the top level and converted to an error message to stderr. This, in turn, allows us to use subprocess.check_call, which eliminates a lot of conditions. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* genrandconfig: move instantiation of SystemInfo downArnout Vandecappelle2017-07-251-3/+3
| | | | | | | | | The SystemInfo class is instantiated globally and passed down to all functions, but it is really only used in fixup_config. So instead, instantiate it there. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* genrandconfig: replace kwargs with explicit argumentsArnout Vandecappelle2017-07-251-28/+16
| | | | | | | | | | | | | kwargs is a left-over from the use of docopt, it's better to use argparse's Namespace object directly. In addition, most functions use just one or two fields of args, so these can just as well be passed directly as arguments to the function. Particularly for outputdir it doesn't make sense to reconstruct it all the time. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* genrandconfig: fix (some) pep8 warningsArnout Vandecappelle2017-07-251-17/+36
| | | | | | | | | | | | | | | | | | | | | | Warnings fixed: E731 do not assign a lambda expression, use a def -> urlopen_closing is defined with a def. urlopen is not used elsewhere so inlined. E302 expected 2 blank lines E501 line too long -> long lines due to a long string are NOT split E701 multiple statements on one line (colon) E722 do not use bare except' -> use "except Exception", so KeyInterrupt and SystemExit are still passed. We never intended to catch those. E741 ambiguous variable name 'l' -> variable name is replaced with the much more descriptive toolchains_csv E271 multiple spaces after keyword E231 missing whitespace after ',' Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* genrandconfig: use subprocess.check_output instead of PopenArnout Vandecappelle2017-07-251-1/+1
| | | | | | | | | | Popen is more complicated and more difficult to understand. check_output raises an exception if the exit code is non-zero, but that's probably what we want if ldd can't be executed. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* utils/genrandconfig: new scriptArnout Vandecappelle2017-07-251-0/+445
This script will be used by the autobuild-run script to generate the configuration to test. It is put in the utils directory because it can also be called directly to allow users to test things. For now, it is a direct copy of the relevant functions from the autobuild-run script. The only changes are: - unneeded import statements are removed; - code/decode wrappers are limited to decode_byte_list; - __main__ handling is added. For now, the only supported arguments are the ones needed for autobuild-run. Follow-up patches will refactor things and also change the way the script is called. In this version, it can be called from the autobuild-run script as: subprocess.call([os.path.join(srcdir, "utils/genrandconfig"), "-i", str(kwargs['instance']), "--toolchains-url", kwargs['toolchains_url']], stdout=log, stderr=log) Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
OpenPOWER on IntegriCloud