buildroot/utils, branch 2019.02-op-build

utils/test-pkg: generate package config if it is not specified

2019-03-27T09:42:52+00:00

It is possible to generate one-line config for the package just by normalize it to the form: BR2_PACKAGE_${pkg_replaced-to_and_uppercase} it simplifes a bit of testing package where no additional config options are needed. Signed-off-by: Vadim Kochan Reviewed-by: Arnout Vandecappelle (Essensium/Mind) Signed-off-by: Thomas Petazzoni (cherry picked from commit a946813dd581ac2e1aea6745edad2bc7ce44e6aa) Signed-off-by: Peter Korsgaard

utils/scanpypi: handle underscores in python packages

2019-03-01T11:37:48+00:00

Some python packages seem to use underscores in inconsistent ways. We can attempt to normalize these by always using dashes for the buildroot name and attempting to autodetect the correct metadata name format. Signed-off-by: James Hilliard Signed-off-by: Peter Korsgaard

utils/scanpypi: protect against zip-slip vulnerability in zip/tar handling

2019-02-12T20:27:35+00:00

For details, see https://github.com/snyk/zip-slip-vulnerability Older python versions do not validate that the extracted files are inside the target directory. Detect and error out on evil paths before extracting .zip / .tar file. Given the scope of this (zip issue was fixed in python 2.7.4, released 2013-04-06, scanpypi is only used by a developer when adding a new python package), the security impact is fairly minimal, but it is good to get it fixed anyway. Reported-by: Bas van Schaik Signed-off-by: Peter Korsgaard

utils/genrandconfig: drop qt4 fixups

2019-02-06T15:30:39+00:00

Now that Qt4 is removed, these fixups can be removed as well. Signed-off-by: Peter Korsgaard Signed-off-by: Arnout Vandecappelle (Essensium/Mind)

utils/check-package: warn about overridden variables

2019-02-05T19:24:57+00:00

For the general case, appending values to variables is OK and also a good practice, like this: |PACKAGE_VAR = value1 |ifeq ... |PACKAGE_VAR += value2 or this, when the above is not possible: |PACKAGE_VAR = value1 |ifeq ... |PACKAGE_VAR := $(PACKAGE_VAR), value2 But this override is an error: |PACKAGE_VAR = value1 |PACKAGE_VAR = value2 as well this one: |ifeq ... |PACKAGE_VAR += value1 |endif |PACKAGE_VAR = value2 And this override is error-prone: |PACKAGE_VAR = value1 |ifeq ... |PACKAGE_VAR = value2 Create a check function to warn about overridden variables. Some variables are likely to have a default value that gets overridden in a conditional, so ignore them. The name of such variables end in _ARCH, _CPU, _SITE, _SOURCE or _VERSION. After ignoring these variable names, there are a few exceptions to this rule in the tree. For them use the comment that disables the check. Signed-off-by: Ricardo Martincoski Cc: Simon Dawson Cc: Thomas Petazzoni Tested-by: Titouan Christophe Signed-off-by: Peter Korsgaard

utils/check-package: handle ifdef/ifndef in .mk files

2019-01-29T15:38:41+00:00

Currently check-package only knows about ifeq/ifneq. Add code to handle ifdef/ifndef as well. Signed-off-by: Ricardo Martincoski Reviewed-by: Arnout Vandecappelle (Essensium/Mind) Signed-off-by: Peter Korsgaard

utils/check-package: allow to disable warning for a line

2019-01-29T15:37:47+00:00

Currently any exceptions for a check function need to be coded into the check-package script itself. Create a pattern that can be used in a comment to make check-package ignore one or more warning types in the line immediately below: # check-package Indent, VariableWithBraces Signed-off-by: Ricardo Martincoski Reviewed-by: Arnout Vandecappelle (Essensium/Mind) Signed-off-by: Peter Korsgaard

check-package: fix Python3 support

2019-01-16T22:14:25+00:00

This script currently uses "/usr/bin/env python" as shebang but it does not really support Python3. Instead of limiting the script to Python2, fix it to support both versions. So change all imports to absolute imports because Python3 follows PEP328 and dropped implicit relative imports. In order to avoid errors when decoding files with the default 'utf-8' codec, use errors="surrogateescape" when opening files, the docs for open() states: "This is useful for processing files in an unknown encoding.". This argument is not compatible with Python2 open() so import 'six' to use it only when running in Python3. As a consequence the file handler becomes explicit, so use it to close() the file after it got processed. This "surrogateescape" is a simple alternative to the complete solution of opening files with "rb" and changing all functions in the lib*.py files to use bytes objects instead of strings. The only case we can have non-ascii/non-utf-8 files being checked by the script are for patch files when the upstream file to be patched is not ascii or utf-8. There is currently one case in the tree: package/urg/0002-urg-gcc6-fix-narrowing-conversion.patch. Signed-off-by: Ricardo Martincoski Cc: Arnout Vandecappelle Reviewed-by: Thomas De Schampheleire Tested-by: Thomas De Schampheleire Signed-off-by: Peter Korsgaard

utils/get-developers: really make it callable from elsewhere than the toplevel directory

2019-01-04T12:23:41+00:00

Commit 62d5558f76a (utils/get-developers: make it callable from elsewhere than the toplevel directory) tried to fix this by passing in the toplevel directory when the DEVELOPERS file is parsed. Unfortunately this is not enough, as E.G. also the paths listed in the patches are relative to the toplevel directory, causing it to not match the entries in the DEVELOPERS file. In concept this can be fixed by also passing the toplevel directory to the Developers class, but the simplest solution is just to chdir to the toplevel Buildroot directory before calling any of the getdeveloperlib functions. This does require us to finish parsing command line arguments (which opens the provided patch files) to not get into trouble with relative paths to patches before chdir'ing / initializing getdeveloperlib. Signed-off-by: Peter Korsgaard

Revert "utils/get-developers: make it callable from elsewhere than the toplevel directory"

2019-01-04T12:23:33+00:00

This reverts commit 62d5558f76a790d1c1b3a1834f1736dc86b6e34f. This actually does not work, as patches contain paths relative to the toplevel directory as well. Signed-off-by: Peter Korsgaard