summaryrefslogtreecommitdiffstats
path: root/support
Commit message (Collapse)AuthorAgeFilesLines
...
* check-host-tar.sh: bump minimum tar version to 1.27 for reproducible tar ↵Peter Korsgaard2018-04-021-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | files with long paths Fixes: http://autobuild.buildroot.net/results/b18/b187e64a61918f17f69588e2355a03286bc5808e tar 1.27 subtly changed the tar format when a GNU long link entry is added (which is done for path elements > 100 characters). The code used to set the permission mode of the link entry to 0: header = start_private_header ("././@LongLink", size, time (NULL)); FILL (header->header.mtime, '0'); FILL (header->header.mode, '0'); FILL (header->header.uid, '0'); FILL (header->header.gid, '0'); FILL (header->header.devmajor, 0); FILL (header->header.devminor, 0); This got dropped in 1.27 by commit df7b55a8f6354e3 (Fix some problems with negative and out-of-range integers), so the settings from start_private_header() are used directly - Which are: TIME_TO_CHARS (t < 0 ? 0 : min (t, MAX_OCTAL_VAL (header->header.mtime)), header->header.mtime); MODE_TO_CHARS (S_IFREG|S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH, header->header.mode); UID_TO_CHARS (0, header->header.uid); GID_TO_CHARS (0, header->header.gid); The end result is that tar >= 1.27 sets mode to 644. The consequence of this is that we create different tar files when long path names are encountered (which often happens when a package downloads a specific sha1 from a git repo) depending on the host tar version used, causing hash mismatches. As a workaround, bump our minimum tar version to 1.27. It would be nicer to only do this if we have packages from bzr/git/hg enabled, but that is an exercise for later. Signed-off-by: Peter Korsgaard <peter@korsgaard.com> Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* download: put most of the infra in dl-wrapperMaxime Hadjinlian2018-04-023-57/+102
| | | | | | | | | | | | | | | | | | | | | | | The goal here is to simplify the infrastructure by putting most of the code in the dl-wrapper as it is easier to implement and to read. Most of the functions were common already, this patch finalizes it by making the pkg-download.mk pass all the parameters needed to the dl-wrapper which in turn will pass everything to every backend. The backend will then cherry-pick what it needs from these arguments and act accordingly. It eases the transition to the addition of a sub directory per package in the DL_DIR, and later on, a git cache. [Peter: drop ';' in BR_NO_CHECK_HASH_FOR in DOWNLOAD macro and swap cd/rm -rf as mentioned by Yann, fix typos] Signed-off-by: Maxime Hadjinlian <maxime.hadjinlian@gmail.com> Tested-by: Luca Ceresoli <luca@lucaceresoli.net> Reviewed-by: Luca Ceresoli <luca@lucaceresoli.net> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* core/pkg-download: change all helpers to use common optionsYann E. MORIN2018-04-029-90/+112
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently all download helpers accepts the local output file, the remote locations, the changesets and so on... as positional arguments. This was well and nice when that's was all we needed. But then we added an option to quiesce their verbosity, and that was shoehorned with a trivial getopts, still keeping all the existing positional arguments as... positional arguments. Adding yet more options while keeping positional arguments will not be very easy, even if we do not envision any new option in the foreseeable future (but 640K ought to be enough for everyone, remember? ;-) ). Change all helpers to accept a set of generic options (-q for quiet and -o for the output file) as well as helper-specific options (like -r for the repository, -c for a changeset...). Maxime: Changed -R to -r for recurse (only for the git backend) Changed -r to -u for URI (for all backend) Change -R to -c for cset (for CVS and SVN backend) Add the export of the BR_BACKEND_DL_GETOPTS so all the backend wrapper can use the same option easily Now all the backends use the same common options. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Maxime Hadjinlian <maxime.hadjinlian@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Reviewed-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* support/scripts/graph-depends: add --flat-list optionGeorge Redivo2018-04-011-7/+19
| | | | | | | | | | | | | | | | | | graph-depends currently spits out a graph in .dot format. However, as part of the upcoming introduction of <pkg>-show-recursive-depends and <pkg>-show-recursive-rdepends, we need graph-depends to be able to display a flat list. Signed-off-by: George Redivo <george.redivo@datacom.ind.br> [Thomas: - Rebase on top of graph-depends changes - Do not display the package name itself in the list, only its dependencies (or reverse dependencies) - Display the result on a single line, instead of one package per line, in order to match what <pkg>-show-depends and <pkg>-show-rdepends are doing today.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* support/scripts/graph-depends: add --quiet optionPeter Korsgaard2018-04-011-1/+3
| | | | | | | | | This will be useful for the upcoming recursive show-depends and show-rdepends features. Signed-off-by: Peter Korsgaard <peter@korsgaard.com> Reviewed-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* support/scripts/graph-depends: use the standard python logging modulePeter Korsgaard2018-04-012-12/+17
| | | | | | | | | Instead of hardcoded sys.stderr.write() calls. No functional change, but allows us to easily implement a quiet option. Signed-off-by: Peter Korsgaard <peter@korsgaard.com> Reviewed-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* support/scripts/graph-depends: use colors instead of coloursThomas Petazzoni2018-04-011-8/+8
| | | | | | | | | | | | | | | | The graph-depends was not very consistent in colors vs. colours: some parts were using colours, some parts were using colors. Let's settle on the US spelling, colors. This change the user-visble option --colours to --colors, but it is unlikely that a lot of users customize the colors through BR2_GRAPH_DEPS_OPTS, so this user interface change is considered reasonable. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* support/scripts/graph-depends: remove global code and most global variablesThomas Petazzoni2018-04-011-145/+144
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The graph-depends script had no main() function, and the main code was actually spread between the function definitions, which was a real mess. This commit moves the global code into a main() function, which allows to more easily follow the flow of the script. The argument parsing code is moved into a parse_args() function. Most of the global variables are removed, and are instead passed as argument when appropriate. This has the side-effect that the print_pkg_deps() function takes a lot of argument, but this is considered better than tons of global variables. The global variables that are removed are: max_depth, transitive, mode, root_colour, target_colour, host_colour, outfile, dict_deps, dict_version, stop_list, exclude_list, arrow_dir. The root_colour/target_colour/host_colour variables are entirely removed, and instead a single colours array is passed, and it's the function using the colors that actually uses the different entries in the array. The way the print_attrs() function determines if we're display the root node is not is changed. Instead of relying on the package name and the mode (which requires passing the root package name, and the mode), it relies on the depth: when the depth is 0, we're at the root node. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* testing/infra/builder: split configure() from build()Ricardo Martincoski2018-04-012-1/+4
| | | | | | | | | | | | | | | | | Some test cases don't use a full build as setup, so split the build() method into configure() and build(). It allows a test case to perform configuration at the setup stage and the build inside the test itself. Call this new method just before build in the BRTest base class, to keep the current behavior for existing test cases. This change will be needed when adding a common class to test the git download infra. Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br> Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* testing/infra/builder: call make with empty envRicardo Martincoski2018-04-013-3/+6
| | | | | | | | | | | | | | | | | | | | | Use an empty environment when calling make, but import PATH so the buildroot tree under test can find binaries from the host machine. Since environment variables are now ignored, move the handling of BR2_DL_DIR to the defconfig to keep the current precedence of -d: BR2_DL_DIR | -d DIR | test downloads | BR downloads ------------+----------+------------------+-------------- unset | unset | [error] | [error] unset | set | in $(DIR) | in $(DIR) set | unset | in $(BR2_DL_DIR) | in $(BR2_DL_DIR) set | set | in $(DIR) | in $(DIR) Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Yann E. MORIN <yann.morin.1998@free.fr> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* toolchain-configs.csv: re-organize for test-pkgThomas Petazzoni2018-04-011-6/+26
| | | | | | | | | | | This commit reorganizes the toolchain-configs.csv so that the first toolchains are a subset of "useful" toolchains to be tested by contributors to validate a package. This subset is the one that will be used by default by test-pkg. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* support/download: keep files downloaded without hashGaël PORTAY2018-04-011-4/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | In the situation where the hash is missing from the hash file, the dl-wrapper downloads the file again and again until the developer specifies the hash to complete the download step. To avoid this situation, the freshly-downloaded file is not removed anymore after a successful download. After this change, the behaviour is as follows: - Hash file doesn't exist, or file is in BR_NO_CHECK_HASH_FOR => always succeeds. - Hash file exists, but file is not present => file is NOT removed, build is terminated immediately (i.e. secondary site is not tried). - Hash file exists, file is present, but hash mismatch => file is removed, secondary site is tried. => If all primary/secondary site downloads or hash checks fail, the build is terminated. Signed-off-by: Gaël PORTAY <gael.portay@savoirfairelinux.com> [Arnout: extend commit log] Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* fs: run packages' filesystem hooks in a copy of target/Yann E. MORIN2018-03-311-10/+13
| | | | | | | | | | | | | | | | | | | | Currently, some packages may register hooks to be run just before and just after the generic tarball image is generated, because they need to prepare the filesystem for read-only or read-write operation. However, this means that, if any of the hooks or the image generation fails, the target directory is left in a dangling, inconsistent state. We fix that by doing a copy of target/, run the hooks on that copy, generate the generic tarball image out of that, and get rid of the copy. This way, we can guarantee consistency of the target directory, and we can even ditch support for post-fs hooks (those that restore target/). Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Arnout Vandecappelle <arnout@mind.be> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* support/check-uniq-files: support weird locales and filenamesYann E. MORIN2018-03-311-6/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, when a filename contains characters not representable in the user's locale, we fail hard, especially when the host python is python3. This is because python2 and python3 handle encoding/decoding strings differently, with python3 presumable doing the right thing, but it breaks on some systems, while python2 presumable does the wrong thing, but it works everywhere. (Just joking, obviously...) Part of the issue being that the csv reader in python2 is broken with UTF8. We fix the issue by ditching the csv reader, and simply read the file in binary mode, manually partitioning the lines on the first comma. Then, we use the binary-encoded (really, un-encoded) package names and filenames as values and keys, respectively. Finally, for each filename or package we need to print, we try to decode them with the defaults for the user settings, but catch any decoding exception and fall back to dumping the raw, binary values. Which codec is used by default differs between Python version, but in all cases something sane is printed at least. Thanks a lot to Arnout for the live help doing this patch. :-) Reported-by: Jaap Crezee <jaap@jcz.nl> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Jaap Crezee <jaap@jcz.nl> [Arnout: commit log improvement] Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* support/config-fragments/autobuild: fix SSP in br-nios2-glibcThomas Petazzoni2018-03-311-0/+1
| | | | | | | | | | | | | | | | Commit c8680956819fae8776d7bd6d1f0e67a7b6436672 ("toolchain: fix detection of SSP support") fixed the SSP check so that it does the correct thing for nios2 toolchains. While this commit fixed the description of the Sourcery NIOSII toolchain, it didn't fix the description for the autobuilders of the br-nios2-glibc toolchain, causing some build failures. This commit adjusts br-nios2-glibc.config to indicate that the toolchain doesn't have SSP support. Fixes: http://autobuild.buildroot.net/results/6c44e328b7bffd8474d29d5bdf1ea109ec15f4ad/ Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* core: kill DEPENDENCIES_HOST_PREREQThomas Petazzoni2018-03-251-8/+2
| | | | | | | | | Now that DEPENDENCIES_HOST_PREREQ is no longer used anywhere, we can kill it. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* package/pkg-generic: handle host-ccache as a regular dependencyThomas Petazzoni2018-03-251-4/+0
| | | | | | | | | | | | | | | | | | This moves the host-ccache dependency handling from DEPENDENCIES_HOST_PREREQ to a proper package dependency. When BR2_CCACHE=y, we add host-ccache as a regular dependency of all packages except: - The extractor packages host-tar, host-xz and host-lzip - host-ccache itself - host-skeleton, because all packages depend on it Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Tested-by: Matt Weber <matthew.weber@rockwellcollins.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* package/pkg-generic: handle host-lzip as an extract dependencyThomas Petazzoni2018-03-251-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This moves the host-lzip dependency handling from DEPENDENCY_HOST_PREREQ to an extract dependency. To achieve that, check-host-lzip.mk fills in the BR2_LZIP_HOST_DEPENDENCY with host-lzip if building a host-lzip is needed. The name BR2_LZIP_HOST_DEPENDENCY has been chosen because it matches the name BR2_CMAKE_HOST_DEPENDENCY already used in check-host-cmake.mk. The BR2_LZIP_HOST_DEPENDENCY is added to all packages, except: - host-lzip, because we would otherwise depend on ourself. - host-tar, because lzip itself is delivered as a tarball, so we need to have host-lzip depend on host-tar, and not host-tar depend on host-lzip - host-skeleton, because we need to have host-lzip depend on host-skeleton, and not the opposite. We also mutually exclude host-lzip and host-xz from dependending on each other, to avoid a circular dependency. In addition, we modify lzip.mk to explicitly build host-lzip without ccache. We generally took the approach of building host-ccache *after* all the extractors have been built. [Peter: fix s/host-tar/host-lzip/ typo, fix s/xz/lzip/ typo] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* package/pkg-generic: handle host-xz as an extract dependencyThomas Petazzoni2018-03-251-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This moves the host-xz dependency handling from DEPENDENCY_HOST_PREREQ to an extract dependency. To achieve that, check-host-xz.mk fills in the BR2_XZ_HOST_DEPENDENCY with host-xz if building a host-xz is needed. The name BR2_XZ_HOST_DEPENDENCY has been chosen because it matches the name BR2_CMAKE_HOST_DEPENDENCY already used in check-host-cmake.mk. The BR2_XZ_HOST_DEPENDENCY is added to all packages, except: - host-xz, because we would otherwise depend on ourself. - host-tar, because xz itself is delivered as a tarball, so we need to have host-xz depend on host-tar, and not host-tar depend on host-xz - host-skeleton, because we need to have host-xz depend on host-skeleton, and not the opposite. In addition, we modify xz.mk to explicitly build host-xz without ccache. We generally took the approach of building host-ccache *after* all the extractors have been built. [Peter: fix s/host-tar/host-xz/ typo] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* package/pkg-generic: handle host-tar as an extract dependencyThomas Petazzoni2018-03-251-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This moves the host-tar dependency handling from DEPENDENCY_HOST_PREREQ to an extract dependency. To achieve that, check-host-tar.mk fills in the BR2_TAR_HOST_DEPENDENCY variable with host-tar if building a host-tar is needed. The name BR2_TAR_HOST_DEPENDENCY has been chosen because it matches the name BR2_CMAKE_HOST_DEPENDENCY already used in check-host-cmake.mk. The BR2_TAR_HOST_DEPENDENCY is added to all packages, except host-tar itself (obviously) and host-skeleton, because we depend on host-skeleton to install host-tar properly in HOST_DIR. In addition, we modify tar.mk to explicitly build host-tar without ccache: since ccache source code is available as a tarball, ccache will obviously depend on host-tar if the system tar is insufficient. Finally, to make things really clean, we also add $(BR2_TAR_HOST_DEPENDENCY) to the dependencies of the tar filesystem format, since it requires tar, so we'd better make sure we have a suitable tar. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Tested-by: Matt Weber <matthew.weber@rockwellcollins.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* support/dockerfile: install flake8Ricardo Martincoski2018-03-131-1/+10
| | | | | | | | | | | | | | | | | | | Use the latest version of the tool because it is actively maintained. But use a fixed version of the tool and its dependencies to get stable results. It can be manually bumped from time to time. Before installing any Python packages, ensure pip, setuptools, and wheel are up to date as recommended in the docs [1]. [1] https://packaging.python.org/tutorials/installing-packages/ Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Peter Korsgaard <peter@korsgaard.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Yann E. MORIN <yann.morin.1998@free.fr> Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* test_python_cryptography: fix code styleRicardo Martincoski2018-03-131-10/+13
| | | | | | | | | | | | | Fix these warnings: E122 continuation line missing indentation or outdented E301 expected 1 blank line, found 0 E302 expected 2 blank lines, found 1 F401 'os' imported but unused Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Cc: Yegor Yefremov <yegorslists@googlemail.com> Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* support/check-bin-arch: exclude kernel modules for merged /usrYann E. MORIN2018-03-071-0/+1
| | | | | | | | | | | When using a merged /usr, the kernel module path is really /usr/lib/modules, as /lib is a symlink to usr/lib . Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Peter Korsgaard <peter@korsgaard.com> Reviewed-by: Peter Korsgaard <peter@korsgaard.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* spport/check-bin-arch: accept arbitrary per-package ignore pathsYann E. MORIN2018-03-071-22/+34
| | | | | | | | | | | | | | | Some packages (mostly, out-of-tree) may want to install binary blobs for another architecture, outside the locations we currently exclude, like in /opt or whatever... Add support in check-bin-arch to accept any arbitrary location, that individual package can each request to excude from the check, when they are installed. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Peter Korsgaard <peter@korsgaard.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* Merge branch 'next'Peter Korsgaard2018-03-055-1/+41
|\ | | | | | | Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
| * support/testing: add python-cryptography testsYegor Yefremov2018-02-061-0/+31
| | | | | | | | | | | | Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com> Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
| * support/config-fragments: remove bogus BR2_JLEVEL optionThomas Petazzoni2018-02-051-1/+0
| | | | | | | | | | | | | | There is no reason to have a BR2_JLEVEL option in such toolchain defconfigs. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
| * support/config-fragments: add internal defconfigs for glibc and muslThomas Petazzoni2018-02-053-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | While we have several defconfigs building internal toolchains with uClibc, we don't have any building internal toolchain with glibc and musl. However, having such defconfigs is nice when we bump the C library version, in order to immediately get feedback on build failures. Note that while the ARC internal defconfig uses glibc, it uses the special ARC glibc version, so it doesn't test version bumps of the upstream glibc C library. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* | Update for 2018.022018.02Peter Korsgaard2018-03-041-1/+1
| | | | | | | | Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* | support/tests: enhance the runtime systemd testsYann E. MORIN2018-03-041-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Recent systemd bump has broken DBus dameon and DBus applications can no longer find the daemon. So we want to catch those kind of failures early. We also want to check that the system as a whole is stable: no unit should be failed. Finally, ensure that we can read the jounrnal, even when we are doing our tricks on read-only systems. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* | core: drop no-longer used C.UTF-8 locale optionYann E. MORIN2018-03-041-13/+0
| | | | | | | | | | | | | | | | | | | | | | | | Its use has been globbed into the more generic BR2_NEEDS_HOST_UTF8_LOCALE option now. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Peter Korsgaard <peter@korsgaard.com> Cc: Julius Kriukas <julius@kriukas.lt> Cc: Christian Stewart <kidovate@gmail.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* | support/dependencies: unbreak check for UTF-8 localeYann E. MORIN2018-03-041-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Although the UTF-8 locales in mainstream distributions all are suffixed with just 'utf8', the nomenclature is a bit ambiguous with the way they are to be specified with the various LC_* variables, suffixed there with 'UTF-8'. Also, POSIX, ISO, and IEC do not enforce any specific suffix in LC_* variables: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_02 """ If the locale value has the form: language[_territory][.codeset] it refers to an implementation-provided locale, where settings of language, territory, and codeset are implementation-defined. """ To avoid any confusion, use a regexp that is a bit more lax when matching locales. Also, quote the regexp, so that the '?' and '$' are not interpreted by the shell. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Peter Korsgaard <peter@korsgaard.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* | support/dependencies/dependencies.sh: check for Python argparse moduleThomas Petazzoni2018-03-031-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The script support/scripts/check-uniq-files uses the argparse Python module. In most recent Python versions (starting with 2.7), the argparse module is part of the standard library, and we already check for the availability of Python in support/dependencies/dependencies.sh. However, when running on an ancient distribution with Python 2.6, the argparse module is not part of the Python standard library, but available as an external module. Without this module, the build fails, because check-uniq-files, which is used in target-finalize, fails to run. To avoid this failure, this commit adds a check in support/dependencies/dependencies.sh to verify that the argparse module is available. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* | support/scripts/check-uniq-files: add indices in format stringThomas Petazzoni2018-03-031-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Using {} in format strings is only supported in sufficiently recent Python versions. Python 2.6 doesn't support this, and only format strings with numbered arguments: {0}, {1}, etc. Python 2.7: $ python -c 'print("foo {}".format(12))' foo 12 $ python -c 'print("foo {0}".format(12))' foo 12 Python 2.6: $ python -c 'print("foo {}".format(12))' Traceback (most recent call last): File "<string>", line 1, in <module> ValueError: zero length field name in format $ python -c 'print("foo {0}".format(12))' foo 12 Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* | systemd: switch to C.UTF-8 locale when buildingJulius Kriukas2018-03-021-0/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When BR2_REPRODUCIBLE is enabled or host uses non UTF-8 capable locale building systemd fails with an error: [1/1080] Generating systemd.bg.catalog with a meson_exe.py custom command. FAILED: catalog/systemd.bg.catalog /buildroot/output/host/bin/python3 /buildroot/output/host/bin/meson --internal exe /buildroot/output/build/systemd-236/build/meson-private/meson_exe_sed_232a0623cc7ce2cd67ec72ed784b76307102ed76.dat Warning: You are using 'ANSI_X3.4-1968' which is not a Unicode-compatible locale. You might see errors if you use UTF-8 strings as filenames, as strings, or as file contents. Please switch to a UTF-8 locale for your platform. ... UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 1079: ordinal not in range(128) package/pkg-generic.mk:247: recipe for target '/buildroot/output/build/systemd-236/.stamp_built' failed make: *** [/buildroot/output/build/systemd-236/.stamp_built] Error 1 This patch changes default host system locale from C to C.UTF-8 when building systemd package to fix this issue. It also introduces BR2_NEEDS_HOST_C_UTF8_LOCALE flag that checks if this locale is available on the host system. If locale is not available error message is show and build process is stopped. Signed-off-by: Julius Kriukas <julius@kriukas.lt> [Thomas: use C.UTF-8 instead of en_US.UTF-8.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* | check-host-tar.sh: blacklist tar 1.30+Peter Korsgaard2018-02-271-9/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Tar 1.30 changed the --numeric-owner output for filenames > 100 characters, leading to hash mismatches for the tar archives we create ourselves from git. This is really a fix for a bug in earlier tar versions regarding deterministic output, so it is unlikely to be reverted in later versions. For more details, see: http://lists.busybox.net/pipermail/buildroot/2018-January/211222.html To work around this issue, blacklist tar 1.30+ similar to how we do it for pre-1.17 versions so Buildroot falls back to building host-tar. Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* | dependencies.mk: check for valid host-tar before other host dependenciesPeter Korsgaard2018-02-271-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | host-{cmake,lzip,xz} needs host-tar to extract their source code tarball, so we need to ensure that host-tar gets added to DEPENDENCIES_HOST_PREREQ before these in case they are both needed, otherwise the tools will fail to extract. With the upcoming change to blacklist modern tar versions this situation is likely to trigger more often. The real solution to this issue is the <foo>_EXTRACT_DEPENDENCIES rework, but that series is a bit too intrusive to add this close to 2018.02, so therefore this hack. Signed-off-by: Peter Korsgaard <peter@korsgaard.com> Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* | check-bin-arch: skip /lib/modules to allow 32-bit userland on 64-bit archThomas De Schampheleire2018-02-251-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The script check-bin-arch fails as follows on a config for PowerPC e6500 (64-bit CPU) with BR2_ARCH="powerpc" (32-bit userland desired): ERROR: architecture for "/lib/modules/..../lib/libcrc32c.ko" is "PowerPC64", should be "PowerPC" This situation is perfectly acceptable: the kernel is 64-bit and so are its modules, even though userland is 32-bit. To keep check-bin-arch and its caller simple, just skip /lib/modules/ entirely, like is done for /lib/firmware and some others. Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* | support/testing: fix job check-gitlab-ci.ymlRicardo Martincoski2018-02-131-6/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently 'run-tests -l' is broken. It breaks 'make .gitlab-ci.yml' that in turn breaks the job in GitLab. TestRustBase is not a test case by itself, so it can't have a method with the name starting with "test_" otherwise nose2 assumes it is a test case. Move the test_run method from the base class to the derived classes. While at it, update .gitlab-ci.yml with the new test cases. Fixes: https://gitlab.com/buildroot.org/buildroot/-/jobs/52000035 Reported-by: Yegor Yefremov <yegorslists@googlemail.com> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Cc: Eric Le Bihan <eric.le.bihan.dev@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Yann E. MORIN <yann.morin.1998@free.fr> Cc: Yegor Yefremov <yegorslists@googlemail.com> Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* | support/dependencies: add check for c++ multilibGaël PORTAY2018-02-062-1/+9
|/ | | | | | | | | | | | Some packages build C++ 32bits host-tools and need the g++-multilib to be installed on the build machine. As example, qt5webengine builds a C++ host-tool when target is 32bits. Add the check for g++-multilib to the dependencies script; and update the Dockerfile to install g++-multilib package. Signed-off-by: Gaël PORTAY <gael.portay@savoirfairelinux.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* support/testing: add tests for RustEric Le Bihan2018-02-051-0/+107
| | | | | | | | | | | | | | To test the support for the Rust language, the following tests are added: - building Rust compiler and Cargo from source. - installing a pre-built Rust compiler and building Cargo from source. For each test, a Rust test program is built and installed in the root file system of a ARM vexpress QEMU system. The test is declared OK if the program can be run properly from the test system. Signed-off-by: Eric Le Bihan <eric.le.bihan.dev@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* support/dockker: create and push tagged imagesYann E. MORIN2018-02-041-3/+3
| | | | | | | | | | | | | | Currently, we refer to the latest version of the image, which means we can't guarantee any reproducibility. Also, it measn we can't have a separate images for the maintenance branches (especially the LTS) and master. Update the comment in the Dockerfile to create and push tagged images. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Peter Korsgaard <peter@korsgaard.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* support/docker: use a fixed Debian snapshotYann E. MORIN2018-02-042-0/+11
| | | | | | | | | | | | | Since we're now using a specific base image tag, we need to also use a specific, stable repository to get additional packages from for this image. As such, use the Debian snapshot that matches the base image. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Peter Korsgaard <peter@korsgaard.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* support/docker: use a known base distributionYann E. MORIN2018-02-041-1/+1
| | | | | | | | | | | | | | | Currently, we are using debian:stable, which is subject to change with time, as new stable versions of Debian are released/updated. Use the latest tagged stable release, stretch-20171210 as of today, as the base distribution to use. This will ease reproducible builds in the future. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Peter Korsgaard <peter@korsgaard.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* support/docker: don't be silent when setting upYann E. MORIN2018-02-041-4/+4
| | | | | | | | | | | This image is not built very often, and when it is, it is important to see what's going on, so don't be silent when installing packages from the distro, and since that can take a bit of time it thus serves as progress report... Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Arnout Vandecappelle <arnout@mind.be> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* support/docker: limit the number of layersYann E. MORIN2018-02-041-15/+15
| | | | | | | | | | | | | | | | | | The official documentation [0] suggests limiting the number of layers generated from a dockerfile. A layer is created for each RUN (and COPY and ADD) command. But we are only ever interested in the final image, so the intermediate layers are useless to us. Limit the number of RUN commands to limit the number of generated layers. [0] https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#minimize-the-number-of-layers Reported-by: Peter Korsgaard <peter@korsgaard.com> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Peter Korsgaard <peter@korsgaard.com> Cc: Arnout Vandecappelle <arnout@mind.be> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* testing/tests/boot/test_atf: fix code styleRicardo Martincoski2018-01-291-61/+63
| | | | | | | | | | | Fix these warnings: E122 continuation line missing indentation or outdented E302 expected 2 blank lines, found 1 F401 'os' imported but unused Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* size-stats: fix code styleRicardo Martincoski2018-01-291-1/+7
| | | | | | | | | | | | Fix these warnings: E302 expected 2 blank lines, found 1 E305 expected 2 blank lines after class or function definition, found 1 E713 test for membership should be 'not in' Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* pycompile: fix code styleRicardo Martincoski2018-01-291-0/+3
| | | | | | | | | | | | Fix these warnings: E302 expected 2 blank lines, found 1 E305 expected 2 blank lines after class or function definition, found 1 Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Yegor Yefremov <yegorslists@googlemail.com> Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* graph-build-time: fix code styleRicardo Martincoski2018-01-291-9/+15
| | | | | | | | | | | | | | | | Fix these warnings: E201 whitespace after '[' E202 whitespace before ']' E302 expected 2 blank lines, found 1 E305 expected 2 blank lines after class or function definition, found 1 Ignore these warnings: E402 module level import not at top of file Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Yann E. MORIN <yann.morin.1998@free.fr> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
OpenPOWER on IntegriCloud