summaryrefslogtreecommitdiffstats
path: root/support/scripts
Commit message (Collapse)AuthorAgeFilesLines
* pkg-autotools: generic configure fix for powerpc64Sam bobroff2016-12-051-0/+47
| | | | | | | | | | | | | | | | | | | Many (100+) packages supported by buildroot contain old configure scripts (or build them from old versions of autotools) that are unable to determine how to link shared libraries on powerpc64 and powerpc64le. This causes that test to erroneously fail on toolchains that are not "bi-endian" (which is the case for toolchains built by buildroot), which causes configure to build static libraries instead of dynamic ones. Although these builds succeed, they tend to cause linker failures in binaries later linked against them. Because affected configure files can be discovered automatically, this patch introduces a hook (enabled only when building for powerpc64 and powerpc64le) that uses a script to scan and fix each package. Signed-off-by: Sam Bobroff <sam.bobroff@au1.ibm.com> Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* core/br2-external: restore compatibility with old distrosYann E. MORIN2016-11-231-17/+17
| | | | | | | | | | | | | | | | | | | | Currently, the br2-external script uses bash-4's associative arrays. However, some oldish enterprise-class distros like RHEL5 still use bash-3.1 which lacks associative arrays. We restore compatibility with those oldish distros using 'eval' to emulate associative arrays, as suggested by Arnout. Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com> Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Max Filippov <jcmvbkbc@gmail.com> Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* get-developers: fix parentheses for printGaël PORTAY2016-11-051-1/+1
| | | | | | | | | | | | | Python3 complains about missing parentheses. $ ./support/scripts/get-developers File "./support/scripts/get-developers", line 45 print f ^ SyntaxError: Missing parentheses in call to 'print' Signed-off-by: Gaël PORTAY <gael.portay@savoirfairelinux.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* linux: add ev3dev extensionDavid Lechner2016-11-021-0/+1
| | | | | | | | This adds an ev3dev Linux drivers extension that provides Linux kernel drivers for LEGO MINDSTORMS EV3 from the ev3dev project. Signed-off-by: David Lechner <david@lechnology.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* core/graph-depends: add option to graph reverse dependenciesYann E. MORIN2016-10-251-2/+16
| | | | | | | | | | | | | | Now that we can dump the reverse dependencies of a package, add the ability to graph those. It does not make sense to do a full reverse graph, as it would be semantically equivalent to the direct graph. So we only provide a per-package reverse graph. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* scripts/get-developers: correct type of patches argumentRahul Bedarkar2016-10-252-15/+14
| | | | | | | | | | | | | | | Current type for 'patches' argument is str. It supposed to only contain names of files. If we specify FileType as type, then we don't need to open file ourself and it allows script to read patch from standard input as well. e.g. $ git show -1 | ./support/scripts/get-developers - Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Signed-off-by: Rahul Bedarkar <rahul.bedarkar@imgtec.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* get-developers: use --cc instead of --to for developersRahul Bedarkar2016-10-191-1/+1
| | | | | | | | | Output of get-developers script in our manual uses --cc for developers, but actual output of get-developers script uses --to. This patch makes code consistent with documentation, by using --cc for developers. Signed-off-by: Rahul Bedarkar <rahul.bedarkar@imgtec.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* core/br2-external: fix link to manual for converting old treesYann E. MORIN2016-10-191-1/+1
| | | | | | | | | | | To be noted: that link will only be valid once we have a released manual. In the meantime, it's accessible on the nightly manual: http://nightly.buildroot.org/#br2-external-converting Reported-by: Benoît Allard <benoit.allard@greenbone.net> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Benoît Allard <benoit.allard@greenbone.net> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* core/br2-external: export paths and descriptionsYann E. MORIN2016-10-191-2/+2
| | | | | | | | | | | | | | | | | | | | | Now that we support multiple br2-external trees, BR2_EXTERNAL is no longer exported in the environment. This means that post-build scripts in a br2-external tree can no longer find their own files (well, they could re-invent the path by stripping their known-relative path, but that'd be ugly, especially since we can very well provide it). Export the path for each br2-external trees as environment variables. Do so for the description as well, as a courtesy. Also, re-order variable definitions to be more logical: first, purely internal variables, then exported variables. Reported-by: Benoît Allard <benoit.allard@greenbone.net> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Benoît Allard <benoit.allard@greenbone.net> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* core: support description for br2-external treesYann E. MORIN2016-10-161-3/+9
| | | | | Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* core: add support for multiple br2-external treesYann E. MORIN2016-10-161-40/+75
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, we only support at most one br2-external tree. Being able to use more than one br2-external tree can be very useful. A use-case would be for having a br2-external to contain the basic packages, basic board defconfigs and board files, provided by one team responsible for the "board-bringup", while other teams consume that br2-external as a base, and complements it each with their own set of packages, defconfigs and extra board files. Another use-case would be for third-parties to provide their own Buildroot packaging in a br2-external tree, along-side the archives for their stuff. Finally, another use-case is to be able to add FLOSS packages in a br2-external tree, and proprietary packages in another. This allows to not touch the Buildroot tree at all, and still be able to get in compliance by providing only that br2-external tree(s) that contains FLOSS packages, leaving aside the br2-external tree(s) with the proprietary bits. What we do is to treat BR2_EXTERNAL as a colon-separated (space- separated also work, and we use that internally) list of paths, on which we iterate to construct: - the list of all br2-external names, BR2_EXTERNAL_NAMES, - the per-br2-external tree BR2_EXTERNAL_$(NAME) variables, which point each to the actual location of the corresponding tree, - the list of paths to all the external.mk files, BR2_EXTERNAL_MKS, - the space-separated list of absolute paths to the external trees, BR2_EXTERNAL_DIRS. Once we have all those variables, we replace references to BR2_EXTERNAL with either one of those. This cascades into how we display the list of defconfigs, so that it is easy to see what br2-external tree provides what defconfigs. As suggested by Arnout, tweak the comment from "User-provided configs" to "External configs", on the assumption that some br2-external trees could be provided by vendors, so not necessarily user-provided. Ditto the menu in Kconfig, changed from "User-provided options" to "External options". Now, when more than one br2-external tree is used, each gets its own sub-menu in the "User-provided options" menu. The sub-menu is labelled with that br2-external tree's name and the sub-menu's first item is a comment with the path to that br2-external tree. If there's only one br2-external tree, then there is no sub-menu; there is a single comment that contains the name and path to the br2-external tree. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Romain Naour <romain.naour@openwide.fr> Cc: Julien CORJON <corjon.j@ecagroup.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* docs/manual: add appendix to convert old br2-external treesYann E. MORIN2016-10-161-1/+6
| | | | | | | | | | | Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Samuel Martin <s.martin49@gmail.com> Cc: Romain Naour <romain.naour@openwide.fr> Cc: Julien CORJON <corjon.j@ecagroup.com> [Peter: slightly reword] Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* core: introduce per br2-external NAMEYann E. MORIN2016-10-161-9/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This unique NAME is used to construct a per br2-external tree variable, BR2_EXTERNAL_$(NAME)_PATH, which contains the path to the br2-external tree. This variable is available both from Kconfig (set in the Kconfig snippet) and from the .mk files. Also, display the NAME and its path as a comment in the menuconfig. This will ultimately allow us to support multiple br2-external trees at once, with that NAME (and thus BR2_EXTERNAL_$(NAME)) uniquely defining which br2-external tree is being used. The obvious outcome is that BR2_EXTERNAL should now no longer be used to refer to the files in the br2-external tree; that location is now known from the BR2_EXTERNAL_$(NAME)_PATH variable instead. This means we no longer need to expose, and must stop from from exposing BR2_EXTERNAL as a Kconfig variable. Finally, this also fixes a latent bug in the pkg-generic infra, where we would so far always refer to BR2_EXTERNAL (even if not set) to filter the names of packages (to decide whether they are a bootloader, a toolchain or a simple package). Note: since the variables in the Makefile and in Kconfig are named the same, the one we computed early on in the Makefile will be overridden by the one in .config when we have it. Thus, even though they are set to the same raw value, the one from .config is quoted and, being included later in the Makefile, will take precedence, so we just re-include the generated Makefile fragment a third time before includeing the br2-external's Makefiles. That's unfortunate, but there is no easy way around that as we do want the two variables to be named the same in Makefile and Kconfig (and we can't ask the user to un-quote that variable himself either), hence this little dirty triple-inclusion trick. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Romain Naour <romain.naour@openwide.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* core/br2-external: validate even moreYann E. MORIN2016-10-161-0/+6
| | | | | | | | | | A br2-external tree must provide external.mk and Config.in. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Romain Naour <romain.naour@openwide.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* core: offload handling of BR2_EXTERNAL into the scriptYann E. MORIN2016-10-161-8/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, we treat the case where we have no br2-external tree (BR2_EXTERNAL is empty) differently from the case where we do have one (BR2_EXTERNAL is not empty). There is now no reason to treat those two cases differently: - the kconfig snippet is always generated appropriately (i.e. it would include the br2-external tree if set, or include nothing otherwise); - we no longer have a dummy br-external tree either. Also, the Makefile code to handle BR2_EXTERNAL is currently quite readable if at least a little bit tricky. However, when we're going to add support for using multiple br2-external trees simultaneously, this code would need to get much, much more complex. To keep the Makefile (rather) simple, offload all of the handling of BR2_EXTERNAL to the recently added br2-external helper script. However, because of Makefiles idiosyncracies, we can't use a rule to generate that Makefile fragment. Instead, we use $(shell ...) to call the helper script, and include the fragment twice: once before the $(shell ...) so we can grab a previously defined BR2_EXTERNAL value, a second time to use the one passed on the command line, if any. Furthermore, we can't error out (e.g. on non-existent br2-external tree) directly from the fragment or we'd get that error on subsequent calls, with no chance to override it even from command line. Instead, we use a variable in which we store the error, set it to empty before the second inclusion, so that only the one newly generated, if any, is taken into account. Since we know the script will always be called from Makefile context first, we know validation will occur in Makefile context first. So we can assume that, if there is an error, it will be detected in Makefile context. Consequently, if the script is called to generate the kconfig fragment, validation has already occured, and there should be no error. So we change the error function to generate Makefile code, so that errors are caught as explained above. Lastly, when the value of BR2_EXTERNAL changes, we want to 'forget' about the previous value of the BR2_EXTERNAL_MK variable, especially in the case where BR2_EXTERNAL is now set to empty, so that we do not try to include it later. That's why we first generate empty version of BR2_EXTERNAL_MK, and then assign it the new value, if any. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Romain Naour <romain.naour@openwide.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* core: get rid of our dummy br2-external treeYann E. MORIN2016-10-161-3/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | Now that we generate a kconfig snippet, we can conditionally include the BR2_EXTERNAL's Config.in only when BR2_EXTERNAL is supplied by the user, which means our empty/dummy Config.in is no needed. As for external.mk, we can also include it only when BR2_EXTERNAL is supplied by the user, which means our empty/dummy external.mk is no longer needed. Ditch both of those files, and: - only generate actual content in the Kconfig snippet when we actually do have a BR2_EXTERNAL provided by the user (i.e. BR2_EXTERNAL is not empty); - add a variable that contains the path to the external.mk provided by the user, or empty if none, and include the path set in that variable (make can 'include' nothing without any problem! ;-) ) Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Romain Naour <romain.naour@openwide.fr> Cc: Julien CORJON <corjon.j@ecagroup.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* core: do not hard-code inclusion of br2-external in KconfigYann E. MORIN2016-10-161-0/+88
| | | | | | | | | | | | | | | | | | | | | | Move the inclusion of br2-external's Config.in to the generated kconfig snippet. This will ultimately allow us to use more than one br2-external tree. Offload the "User-provided options" menu to the generated Kconfig snippet. We can also move the definition of the Kconfig-version of BR2_EXTERNAL into this snippet. We introduce an extra check that was not present in the previous code, to check that we do have permission on that directory. Prevciously, it was handled as a side effect of not being able to cd into there, but it is cleaner to check it expressly. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Romain Naour <romain.naour@openwide.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* support/scripts/get-developers: add new scriptThomas Petazzoni2016-09-212-0/+284
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This script, and its companion library, is more-or-less Buildroot's equivalent to the kernel get_maintainer.pl script: it allows to get the list of developers to whom a set of patches should be sent to. To do so, it first relies on a text file, named DEVELOPERS, at the root of the Buildroot source tree (added in a followup commit) to list the developers and the files they are interested in. The DEVELOPERS file's format is simple: N: Firstname Lastname <email> F: path/to/file F: path/to/another/file This allows to associate developers with the files they are looking after, be they related to a package, a defconfig, a filesystem image, a package infrastructure, the documentation, or anything else. When a directory is given, the tool assumes that the developer handles all files and subdirectories in this directory. For example "package/qt5/" can be used for the developers looking after all the Qt5 packages. Conventional shell patterns can be used, so "package/python-*" can be used for the developers who want to look after all packages matching "python-*". A few files are recognized specially: - .mk files are parsed, and if they contain $(eval $(<something>-package)), the developer is assumed to be looking after the corresponding package. This way, autobuilder failures for this package can be reported directly to this developer. - arch/Config.in.<arch> files are recognized as "the developer is looking after the <arch> architecture". In this case, get-developer parses the arch/Config.in.<arch> to get the list of possible BR2_ARCH values. This way, autobuilder failures for this package can be reported directly to this developer. - pkg/pkg-<infra>.mk are recognized as "the developer is looking after the <infra> package infrastructure. In this case, any patch that adds or touches a .mk file that uses this infrastructure will be sent to this developer. Examples of usage: $ ./support/scripts/get-developers 0001-ffmpeg-fix-bfin-build.patch git send-email--to buildroot@buildroot.org --to "Luca Ceresoli <luca@lucaceresoli.net>" --to "Bernd Kuhls <bernd.kuhls@t-online.de>" $ ./support/scripts/get-developers -p imx-lib Arnout Vandecappelle <arnout@mind.be> Gary Bisson <gary.bisson@boundarydevices.com> $ ./support/scripts/get-developers -a bfin Waldemar Brodkorb <wbx@openadk.org> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* manual: get rid of the lists of packagesYann E. MORIN2016-09-193-4084/+0
| | | | | | | | | | | | | | | | | | | | | | | | | We currently have four lists of packages in the manual: - the non-virtual target packages, - the virtual target packages, - the host packages, - the deprecated features. Those list take more than half of the manual. They do not serve much purpose except to show off. After the recent discussion on the list [0], remove them all. We can now get rid of our biggish and complex generating script (and its companion library kconfiglib). [0] http://lists.busybox.net/pipermail/buildroot/2016-September/171199.html Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Peter Korsgaard <peter@korsgaard.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Samuel Martin <s.martin49@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* size-stats: fix bug when parsing file names with commaThomas Petazzoni2016-08-091-1/+1
| | | | | | | | | | | | | | | | | | | The size-stats script fails when the usb_modeswitch_data is enabled, because this package installs files that contain commas in their name. However, the size-stats script also uses comma as a separator for its CSV files, causing a "ValueError: too many values to unpack" in: pkg, fpath = l.split(",") Fix this by splitting only the two fields that need to be split. The bug was reported by Matthias <porto.rio@gmx.net>, who also suggested a fix. Fixes bug #9136. Reported-by: Matthias <porto.rio@gmx.net> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support: pkg-stats: add efl package in packages statsRomain Naour2016-07-241-1/+0
| | | | | | | | Since efl update to 1.15 version, the efl package is a "real" Buildroot package. It doesn't contain any subdirectories anymore. Signed-off-by: Romain Naour <romain.naour@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* scanpypi: include LICENCE in the list of supported license filesYegor Yefremov2016-07-071-1/+2
| | | | | Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/apply-patches: re-instate set -eYann E. MORIN2016-07-061-1/+3
| | | | | | | | | | | | | | | | | | | As reported by Sébastien Szymanski [1], the apply-patches script doesn't stop if a tar command can't extract an archive. Use "set -e" to exit immediately if a command return an error. Be sure to ignore any expected error: when we check if a patch to be applied has the same basename as an already applied patch, the grep would fail when no such patch was already applied. We should not fail in this case. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Sébastien Szymanski <sebastien.szymanski@armadeus.com> Cc: Romain Naour <romain.naour@openwide.fr> Reviewed-by: Romain Naour <romain.naour@gmail.com> Tested-by: Romain Naour <romain.naour@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* Revert "apply-patches: catch unexpected failure"Thomas Petazzoni2016-07-051-3/+0
| | | | | | | | This reverts commit 9cf1ad6cdb37c5be5772121896f4b13ec42c2b0d. This breaks many packages. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* apply-patches: catch unexpected failureRomain Naour2016-07-051-0/+3
| | | | | | | | | | | | | As reported by Sébastien Szymanski [1], the apply-patches script doesn't stop if a tar command can't extract an archive. Use "set -e" to exit immediately if a command return an error. [1] http://patchwork.ozlabs.org/patch/626196 Signed-off-by: Romain Naour <romain.naour@gmail.com> Cc: Sébastien Szymanski <sebastien.szymanski@armadeus.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* scancpan: improve message when bad host perl versionFrancois Perrad2016-07-041-5/+14
| | | | | | | | | | Signed-off-by: Francois Perrad <francois.perrad@gadz.org> [Thomas: - add comment in scancpan about the version dependency, suggested by Yann E. Morin. - add comment in perl.mk about the need to sync any version change with scancpan, also suggested by Yann E. Morin.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* scanpypi: change hash file commentsYegor Yefremov2016-07-041-3/+2
| | | | | | | | | | sha256 checksum will be computed locally either by scanpypi at package creation or by hand by package updates. Define this checksum as 'computed locally' so that one doesn't need to change this comment by package updates. Also put comments for both md5 and sha256 in one line. Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* scanpypi: fix help text handlingYegor Yefremov2016-07-041-0/+5
| | | | | | | Make sure a help text is terminated with a full stop. Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* scanpypi: rework runtime dependency handlingYegor Yefremov2016-07-011-3/+1
| | | | | | | | | | | | | | In most cases Python's package dependencies found in setup.py are runtime dependencies and hence don't need to be mentioned in *.mk file. Also add '# runtime' tag to select statements in Config.in. __create_mk_requirements() itself is left for future uses (cffi backend handling etc.). Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/apply-patches: bail-out on duplicate patch basenamesYann E. MORIN2016-06-241-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Patches we save can come from various locations: - bundled with Buildroot - downloaded - from one or more global-patch-dir It is possible that two patches lying into different locations have the same basename, like so (first is bundled, second is from an hypothetical global-patch-dir): package/foo/0001-fix-Makefile.patch /path/to/my/patches/foo/0001-fix-Makefile.patch In that case, when running legal-info, we'd save only the second patch, overwriting the first. That would be problematic, because: - either the second patch depends on the first, and thus would no longer apply (this is easy to detect, though), - or the second patch does not depend on the first, and the compliance delivery will not be complete (this is much harder to detect). We fix that by checking that no two patches have the same same basename. If we find that the basename of the patch to be applied collides with that of a previously applied patch, we error out and report the duplicate. The unfortunate side-effect is that existing setups will now break in that situation, but that's a minor, corner-case issue that is easily fixed. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Luca Ceresoli <luca@lucaceresoli.net> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> [Thomas: adjust coding style, fix minor typos in the commit log.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* core/apply-patches: store full path of applied patchesYann E. MORIN2016-06-241-3/+8
| | | | | | | | | | | | | | | | | | | | | | | Currently, we only store the filename of the applied patches. However, we are soon to want to install those patches in the legal-info directory, so we'll have to know where those patches come from. Instead of duplicating the logic to find the patches (bundled, downloaded, from a global patch dir...), just store the full path to each of those patches so we can retrieve them more easily later on. Also always create the list-file, even if empty, so that we need not test for its existence before reading it. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Luca Ceresoli <luca@lucaceresoli.net> Reviewed-by: Luca Ceresoli <luca@lucaceresoli.net> [Tested only with patches in the Buildroot sources] Tested-by: Luca Ceresoli <luca@lucaceresoli.net> Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> [Thomas: used $PWD instead of $(pwd), as suggested by Arnout.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* scanpypi: new utilityDenis THULIN2016-06-111-0/+650
| | | | | | | | | | | | A utility for creating python package from the python package index. It fetches packages info from http://pypi.python.org and generates corresponding packages files. Signed-off-by: Denis THULIN <denis.thulin@openwide.fr> Tested-by: Carlos Santos <casantos@datacom.ind.br> Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com> [Thomas: minor tweaks.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* Merge branch 'next'Peter Korsgaard2016-06-012-0/+59
|\ | | | | | | Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
| * python/python3: globalize *.pyc files compilationYegor Yefremov2016-05-261-0/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, each python package (be it the python interpreter package itself or external python modules) is responsible for compiling its .py into .pyc files. Unfortunately, this is not ideal as some packages only install .py files without compiling them into .pyc files. In this case, if the Buildroot configuration specifies to keep only the .pyc files, the .py files are removed and lost. To address this, this commit changes the logic by making the compilation of .pyc files a global operation: the python interpreter packages register a target finalize hook that is in charge of compiling all installed .py files. The *.pyc generation on a per package basis is disabled in the python-package infrastructure by passing the "--no-compile" option to setup.py. The *.pyc generation for the Python interpreter internal modules is disabled through --disable-pyc-build configure option. A small helper script is used to perform the compilation, the purpose of this script is to abort the compilation process if one of the .py file cannot be compiled. It has been provided by Samuel Martin and integrated into this commit. Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com> Cc: Samuel Martin <s.martin49@gmail.com> [Thomas: - rework for python 3.5 - integrate Samuel proposal that allows to detect compilation failures.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Reviewed-by: Samuel Martin <s.martin49@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
| * support/scripts: add helper to hardlink-or-copyYann E. MORIN2016-05-111-0/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When preparing the legal-info, the source archives are copied in the legal-info/ output directory. When the archives are big, it can take quite a bit of time and unnecessarily uses disk space. When the legal-info output directory is on the same filesystem as the BR2_DL_DIR, we can easily reduce copy time and disk usage by just using hardlins instead of copying. However, the BR2_DL_DIR may be on a different filesystem, so we must fallback to copying in this case Introduce a helper script that copies a source file into a destination directory, by first attempting to hard-link, and falling back to a plain copy in case the hardlink fails. In case the destination already exists, it is forcibly removed first, to avoid clobering any existing target file (and especially any hardlink to it), since cp -f does not remove the destination file, but clobbers it. In some situations, it will be necessary that the destination file is named differently than the source, so if a third argument is specified, it is treated as the basename of the destination file. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Luca Ceresoli <luca@lucaceresoli.net> Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* | support/scripts: fix graph-build-time help textThomas Petazzoni2016-05-241-1/+1
|/ | | | | | | | | | | | The graph-build-time help text currently looks like this: usage: graph-build-time [-h] [--type GRAPH_TYPE] [--order GRAPH_ORDER] [--alternate-colors] [--input OUTPUT] --output OUTPUT Obviously, naming the parameter for --input as OUTPUT is not a very good idea, so this commit fixes that to name it "INPUT", as expected. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/scripts/check-host-rpath: also check HOST_DIR/{bin, sbin}Samuel Martin2016-04-211-1/+1
| | | | | | | | At least syslinux is installing stuff in HOST_DIR/sbin. Cc: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Samuel Martin <s.martin49@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/scripts: fix graph-depends when run with python3Samuel Martin2016-04-181-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Make graph-depends script opening the output file in text mode since only ascii characters will be written. This change fixes the following error occuring when the default host python interpreter is python3: make: Entering directory '/opt/buildroot' Getting targets Getting dependencies for ['toolchain-external', 'toolchain', 'busybox', ...] Getting dependencies for ['host-python3', 'host-pkgconf', 'host-gettext', ...] Getting dependencies for ['host-libxml2', 'host-swig', 'host-m4', ...] Getting version for ['toolchain-external', 'toolchain', 'busybox', ...] Traceback (most recent call last): File "/opt/buildroot/support/scripts/graph-depends", line 425, in <module> outfile.write("digraph G {\n") TypeError: a bytes-like object is required, not 'str' Makefile:807: recipe for target 'graph-depends' failed make[1]: *** [graph-depends] Error 1 Makefile:84: recipe for target '_all' failed make: *** [_all] Error 2 make: Leaving directory '/opt/buildroot' While with python2, adding 'b' to the openning mode has no effect on Linux (c.f. [2]), the above error is expected with python3 (c.f. [1]). Therefore, just open the outfile in default (i.e. text) mode. [1] https://docs.python.org/3/library/functions.html#open [2] https://docs.python.org/2/library/functions.html#open Signed-off-by: Samuel Martin <s.martin49@gmail.com> Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* scancpan: use recommend & test flags only at first levelFrancois Perrad2016-03-151-6/+6
| | | | | | | | Currently, these flags are recursively propagated. This behavior is not expected by users, because it can cause dependencies explosively. Signed-off-by: Francois Perrad <francois.perrad@gadz.org> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* scancpan: handle recommended dependencies as optional packagesFrancois Perrad2016-03-151-3/+22
| | | | | | | | | | | | | | Currently, without the flag -recommend, scancpan takes as dependency only one which has the relationship "requires"; this mode works fine. And, with the flag -recommend, scancpan takes all ones (ie. with relationship "requires" or "recommends") in the same way; this mode never works fine, because it is too simplistic. With this commit, the "not required" dependencies are handled as optional BR package or skipped if a cyclic dependency is detected. Signed-off-by: Francois Perrad <francois.perrad@gadz.org> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* graph-depends: add support for excluding host packagesThomas Petazzoni2016-02-081-2/+8
| | | | | | | | | | | | | | Just like the --stop-on and --exclude options allow to stop on or exclude virtual packages from the list by passing the "virtual" magic value, this commit extends the graph-depends logic to support a "host" magic value for --stop-on and --exclude. This will allow to draw the graph by stopping on host packages, or by excluding host packages. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> [Thomas: minor code beautification suggested by Yann E. Morin.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* graph-depends: fix handling of "virtual" in exclude_listThomas Petazzoni2016-02-081-4/+3
| | | | | | | | | | | | | | | | | | | | | The condition to determine if a virtual package should be excluded from the list due to "virtual" being passed in --exclude is under a loop iterating over each entry of the exclude_list, but it doesn't use the iterator of this list. Indeed, the condition contains: "virtual" in exclude_list which checks automatically if "virtual" was passed in the list. Due to this, there is no need for this check to be within the "for p in exclude_list" iteration. This commit fixes that by moving the check outside of the loop. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/graph-depends: teach it to only check dependenciesYann E. MORIN2016-02-071-0/+10
| | | | | | | | | | Add an option to graph-depends to only do the dependency checks and not generate the dot program. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Samuel Martin <s.martin49@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/graph-depends: detect circular dependenciesYann E. MORIN2016-02-071-0/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, if there is a circular dependency in the packages, the graph-depends script just errors out with a Python RuntimeError which is not caught, resulting in a very-long backtrace which does not provide any hint as what the real issue is (even if "RuntimeError: maximum recursion depth exceeded" is a pretty good hint at it). We fix that by recursing the dependency chain of each package, until we either end up with a package with no dependency, or with a package already seen along the current dependency chain. We need to introduce a new function, check_circular_deps(), because we can't re-use the existing ones: - remove_mandatory_deps() does not iterate, - remove_transitive_deps() does iterate, but we do not call it for the top-level package if it is not 'all' - it does not make sense to use those functions anyway, as they were not designed to _check_ but to _act_ on the dependency chain. Since we've had time-related issues in the past, we do not want to introduce yet another time-hog, so here are timings with the circular dependency check: $ time python -m cProfile -s cumtime support/scripts/graph-depends [...] 28352654 function calls (20323050 primitive calls) in 87.292 seconds Ordered by: cumulative time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.012 0.012 87.292 87.292 graph-depends:24(<module>) 21 0.000 0.000 73.685 3.509 subprocess.py:473(_eintr_retry_call) 7 0.000 0.000 73.655 10.522 subprocess.py:768(communicate) 7 73.653 10.522 73.653 10.522 {method 'read' of 'file' objects} 5/1 0.027 0.005 43.488 43.488 graph-depends:164(get_all_depends) 5 0.003 0.001 43.458 8.692 graph-depends:135(get_depends) 1 0.001 0.001 25.712 25.712 graph-depends:98(get_version) 1 0.001 0.001 13.457 13.457 graph-depends:337(remove_extra_deps) 1717 1.672 0.001 13.050 0.008 graph-depends:290(remove_transitive_deps) 9784086/2672326 5.079 0.000 11.363 0.000 graph-depends:274(is_dep) 2883343/1980154 2.650 0.000 6.942 0.000 graph-depends:262(is_dep_uncached) 1 0.000 0.000 4.529 4.529 graph-depends:121(get_targets) 2883343 1.123 0.000 1.851 0.000 graph-depends:246(is_dep_cache_insert) 9784086 1.783 0.000 1.783 0.000 graph-depends:255(is_dep_cache_lookup) 2881580 0.728 0.000 0.728 0.000 {method 'update' of 'dict' objects} 1 0.001 0.001 0.405 0.405 graph-depends:311(check_circular_deps) 12264/1717 0.290 0.000 0.404 0.000 graph-depends:312(recurse) [...] real 1m27.371s user 1m15.075s sys 0m12.673s The cumulative time spent in check_circular_deps is just below 0.5s, which is largely less than 1% of the total run time. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Samuel Martin <s.martin49@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/graph-depends: add option to specify output fileYann E. MORIN2016-02-071-6/+13
| | | | | | | | | | | | | | | | | | | | | Currently, graph-depends outputs the dotfile program to stdout, and uses stderr to trace the dependencies it is currently looking for. Redirection was done because the output was directly piped into the dot program to generate the final PDF/SVG/... dependency graph, but that meant that an error in the graph-depends script was never caught (because shell pipes only return the final command exit status, and an empty dot program is perfectly valid so dot would not complain). Add an option to tell graph-depends where to store the generated dot program, and keep stdout as the default if not specified. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Samuel Martin <s.martin49@gmail.com> [Thomas: rename metavar from DOT_FILE to OUT_FILE for consistency with the rest of the new option naming.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/scripts: add size-stats-compare scriptThomas De Schampheleire2016-02-071-0/+127
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Leverage the CSV files produces by size-stats (make graph-size) to allow for a comparison of rootfs size between two different buildroot compilations. The script takes the file-size CSV files of two compilations as input, and produces a textual report of the differences per package. Using the -d/--detail flag, the report will show the file size changes instead of package size changes. The -t/--threshold option allows to ignore file size differences smaller or equal than the given threshold (in bytes). Example output is: Size difference per package (bytes), threshold = 0 -------------------------------------------------------------------------------- -8192 busybox 228572 added dmalloc 301584 added jq -------------------------------------------------------------------------------- 521964 TOTAL or with detailed view: Size difference per file (bytes), threshold = 0 -------------------------------------------------------------------------------- -8192 bin/busybox 18152 added usr/bin/jq 39252 added usr/bin/dmalloc 46968 added usr/lib/libdmalloc.so 47288 added usr/lib/libdmallocxx.so 47316 added usr/lib/libdmallocth.so 47748 added usr/lib/libdmallocthcxx.so 283432 added usr/lib/libjq.so.1.0.4 -------------------------------------------------------------------------------- 521964 TOTAL Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support/scripts: drop ancient build-ext3-img scriptPeter Korsgaard2016-02-011-152/+0
| | | | | | | It hasn't been updated since it was added in 2008, and nowadays things kind of stuff should be handled with genimage. Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* support/scripts/apply-patches.sh: fix whitespaceArnout Vandecappelle2016-01-131-25/+25
| | | | | | | | | The apply-patches.sh script was using a mix of tabs and spaces, and some three-space indentation. Normalize everything to four-space indentation. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* support/scripts/size-stats: fix copy/paste error in ArgumentParserThomas Petazzoni2016-01-131-1/+1
| | | | | | | | | | | A copy/paste error in the ArgumentParser() constructor call disclosed the fact that the author of the script has shamefully based his work on the existing graph-build-time script. This commit fixes this mistake, therefore hiding in a better way how size-stats was vampirized from graph-build-time. Reported-by: Thomas De Schampheleire <patrickdepinguin@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* graph-depends: correct is_dep() commentPeter Korsgaard2015-12-291-1/+1
| | | | | | The uncached variant is called is_dep_uncached(), not is_dep_full(). Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
OpenPOWER on IntegriCloud