buildroot/support/dependencies, branch 2018.02.1

check-host-tar.sh: bump minimum tar version to 1.27 for reproducible tar files with long paths

2018-04-08T16:10:29+00:00

Fixes: http://autobuild.buildroot.net/results/b18/b187e64a61918f17f69588e2355a03286bc5808e tar 1.27 subtly changed the tar format when a GNU long link entry is added (which is done for path elements > 100 characters). The code used to set the permission mode of the link entry to 0: header = start_private_header ("././@LongLink", size, time (NULL)); FILL (header->header.mtime, '0'); FILL (header->header.mode, '0'); FILL (header->header.uid, '0'); FILL (header->header.gid, '0'); FILL (header->header.devmajor, 0); FILL (header->header.devminor, 0); This got dropped in 1.27 by commit df7b55a8f6354e3 (Fix some problems with negative and out-of-range integers), so the settings from start_private_header() are used directly - Which are: TIME_TO_CHARS (t < 0 ? 0 : min (t, MAX_OCTAL_VAL (header->header.mtime)), header->header.mtime); MODE_TO_CHARS (S_IFREG|S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH, header->header.mode); UID_TO_CHARS (0, header->header.uid); GID_TO_CHARS (0, header->header.gid); The end result is that tar >= 1.27 sets mode to 644. The consequence of this is that we create different tar files when long path names are encountered (which often happens when a package downloads a specific sha1 from a git repo) depending on the host tar version used, causing hash mismatches. As a workaround, bump our minimum tar version to 1.27. It would be nicer to only do this if we have packages from bzr/git/hg enabled, but that is an exercise for later. Signed-off-by: Peter Korsgaard Acked-by: "Yann E. MORIN" Signed-off-by: Peter Korsgaard (cherry picked from commit cdac332d20d2d1326dee0111e188fa214549122b) Signed-off-by: Peter Korsgaard

core: drop no-longer used C.UTF-8 locale option

2018-03-04T10:59:16+00:00

Its use has been globbed into the more generic BR2_NEEDS_HOST_UTF8_LOCALE option now. Signed-off-by: "Yann E. MORIN" Cc: Thomas Petazzoni Cc: Peter Korsgaard Cc: Julius Kriukas Cc: Christian Stewart Signed-off-by: Peter Korsgaard

support/dependencies: unbreak check for UTF-8 locale

2018-03-04T10:46:48+00:00

Although the UTF-8 locales in mainstream distributions all are suffixed with just 'utf8', the nomenclature is a bit ambiguous with the way they are to be specified with the various LC_* variables, suffixed there with 'UTF-8'. Also, POSIX, ISO, and IEC do not enforce any specific suffix in LC_* variables: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_02 """ If the locale value has the form: language[_territory][.codeset] it refers to an implementation-provided locale, where settings of language, territory, and codeset are implementation-defined. """ To avoid any confusion, use a regexp that is a bit more lax when matching locales. Also, quote the regexp, so that the '?' and '$' are not interpreted by the shell. Signed-off-by: "Yann E. MORIN" Cc: Peter Korsgaard Cc: Thomas Petazzoni Signed-off-by: Peter Korsgaard

support/dependencies/dependencies.sh: check for Python argparse module

2018-03-03T16:18:18+00:00

The script support/scripts/check-uniq-files uses the argparse Python module. In most recent Python versions (starting with 2.7), the argparse module is part of the standard library, and we already check for the availability of Python in support/dependencies/dependencies.sh. However, when running on an ancient distribution with Python 2.6, the argparse module is not part of the Python standard library, but available as an external module. Without this module, the build fails, because check-uniq-files, which is used in target-finalize, fails to run. To avoid this failure, this commit adds a check in support/dependencies/dependencies.sh to verify that the argparse module is available. Signed-off-by: Thomas Petazzoni Acked-by: "Yann E. MORIN" Signed-off-by: Peter Korsgaard

systemd: switch to C.UTF-8 locale when building

2018-03-02T21:44:34+00:00

When BR2_REPRODUCIBLE is enabled or host uses non UTF-8 capable locale building systemd fails with an error: [1/1080] Generating systemd.bg.catalog with a meson_exe.py custom command. FAILED: catalog/systemd.bg.catalog /buildroot/output/host/bin/python3 /buildroot/output/host/bin/meson --internal exe /buildroot/output/build/systemd-236/build/meson-private/meson_exe_sed_232a0623cc7ce2cd67ec72ed784b76307102ed76.dat Warning: You are using 'ANSI_X3.4-1968' which is not a Unicode-compatible locale. You might see errors if you use UTF-8 strings as filenames, as strings, or as file contents. Please switch to a UTF-8 locale for your platform. ... UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 1079: ordinal not in range(128) package/pkg-generic.mk:247: recipe for target '/buildroot/output/build/systemd-236/.stamp_built' failed make: *** [/buildroot/output/build/systemd-236/.stamp_built] Error 1 This patch changes default host system locale from C to C.UTF-8 when building systemd package to fix this issue. It also introduces BR2_NEEDS_HOST_C_UTF8_LOCALE flag that checks if this locale is available on the host system. If locale is not available error message is show and build process is stopped. Signed-off-by: Julius Kriukas [Thomas: use C.UTF-8 instead of en_US.UTF-8.] Signed-off-by: Thomas Petazzoni

check-host-tar.sh: blacklist tar 1.30+

2018-02-27T16:02:26+00:00

Tar 1.30 changed the --numeric-owner output for filenames > 100 characters, leading to hash mismatches for the tar archives we create ourselves from git. This is really a fix for a bug in earlier tar versions regarding deterministic output, so it is unlikely to be reverted in later versions. For more details, see: http://lists.busybox.net/pipermail/buildroot/2018-January/211222.html To work around this issue, blacklist tar 1.30+ similar to how we do it for pre-1.17 versions so Buildroot falls back to building host-tar. Signed-off-by: Peter Korsgaard

dependencies.mk: check for valid host-tar before other host dependencies

2018-02-27T16:01:47+00:00

host-{cmake,lzip,xz} needs host-tar to extract their source code tarball, so we need to ensure that host-tar gets added to DEPENDENCIES_HOST_PREREQ before these in case they are both needed, otherwise the tools will fail to extract. With the upcoming change to blacklist modern tar versions this situation is likely to trigger more often. The real solution to this issue is the _EXTRACT_DEPENDENCIES rework, but that series is a bit too intrusive to add this close to 2018.02, so therefore this hack. Signed-off-by: Peter Korsgaard Reviewed-by: Matt Weber Signed-off-by: Peter Korsgaard

support/dependencies: add check for c++ multilib

2018-02-06T16:21:09+00:00

Some packages build C++ 32bits host-tools and need the g++-multilib to be installed on the build machine. As example, qt5webengine builds a C++ host-tool when target is 32bits. Add the check for g++-multilib to the dependencies script; and update the Dockerfile to install g++-multilib package. Signed-off-by: Gaël PORTAY Signed-off-by: Thomas Petazzoni

dependencies: correct dependencies for top-level parallel build

2018-01-08T22:37:59+00:00

Commit 4932c8a7cc94277b3d53cecd935f725b91cfa9fe introduced the core-dependencies target to make sure that the dependencies.sh script runs before we attempt to compile any host tool, so that the absence of a compiler is properly detected. However, this relied on the left-to-right evaluation of dependencies. This will no longer be true when we enable top-level parallel build. Fix this by letting DEPENDENCIES_HOST_PREREQ depend on core-dependencies. Note that it is not possible to remove the dependencies <- core-dependencies. Indeed, it is possible that DEPENDENCIES_HOST_PREREQ is completely empty, and in that case we still need to check core-dependencies. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) Signed-off-by: Thomas Petazzoni

core-dependencies: remove unneeded HOSTCC

2018-01-08T22:37:54+00:00

dependencies.sh uses HOSTCC_NOCCACHE directly, and this variable is exported from the top-level Makefile, so there is no need to pass HOSTCC to it. HOSTCC is not used at all in dependencies.sh. Thus, we also no longer need to apply the HOSTCC override for core-dependencies. The core-depencies rule doesn't use HOSTCC or HOSTCXX. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) Signed-off-by: Thomas Petazzoni