buildroot/support/scripts, branch 2016.05

support/scripts: fix graph-build-time help text

2016-05-24T21:26:25+00:00

The graph-build-time help text currently looks like this: usage: graph-build-time [-h] [--type GRAPH_TYPE] [--order GRAPH_ORDER] [--alternate-colors] [--input OUTPUT] --output OUTPUT Obviously, naming the parameter for --input as OUTPUT is not a very good idea, so this commit fixes that to name it "INPUT", as expected. Signed-off-by: Thomas Petazzoni

support/scripts/check-host-rpath: also check HOST_DIR/{bin, sbin}

2016-04-21T20:17:36+00:00

At least syslinux is installing stuff in HOST_DIR/sbin. Cc: "Yann E. MORIN" Signed-off-by: Samuel Martin Signed-off-by: Thomas Petazzoni

support/scripts: fix graph-depends when run with python3

2016-04-18T09:34:33+00:00

Make graph-depends script opening the output file in text mode since only ascii characters will be written. This change fixes the following error occuring when the default host python interpreter is python3: make: Entering directory '/opt/buildroot' Getting targets Getting dependencies for ['toolchain-external', 'toolchain', 'busybox', ...] Getting dependencies for ['host-python3', 'host-pkgconf', 'host-gettext', ...] Getting dependencies for ['host-libxml2', 'host-swig', 'host-m4', ...] Getting version for ['toolchain-external', 'toolchain', 'busybox', ...] Traceback (most recent call last): File "/opt/buildroot/support/scripts/graph-depends", line 425, in outfile.write("digraph G {\n") TypeError: a bytes-like object is required, not 'str' Makefile:807: recipe for target 'graph-depends' failed make[1]: *** [graph-depends] Error 1 Makefile:84: recipe for target '_all' failed make: *** [_all] Error 2 make: Leaving directory '/opt/buildroot' While with python2, adding 'b' to the openning mode has no effect on Linux (c.f. [2]), the above error is expected with python3 (c.f. [1]). Therefore, just open the outfile in default (i.e. text) mode. [1] https://docs.python.org/3/library/functions.html#open [2] https://docs.python.org/2/library/functions.html#open Signed-off-by: Samuel Martin Acked-by: Arnout Vandecappelle (Essensium/Mind) Signed-off-by: Thomas Petazzoni

scancpan: use recommend & test flags only at first level

2016-03-15T22:16:33+00:00

Currently, these flags are recursively propagated. This behavior is not expected by users, because it can cause dependencies explosively. Signed-off-by: Francois Perrad Signed-off-by: Peter Korsgaard

scancpan: handle recommended dependencies as optional packages

2016-03-15T22:16:29+00:00

Currently, without the flag -recommend, scancpan takes as dependency only one which has the relationship "requires"; this mode works fine. And, with the flag -recommend, scancpan takes all ones (ie. with relationship "requires" or "recommends") in the same way; this mode never works fine, because it is too simplistic. With this commit, the "not required" dependencies are handled as optional BR package or skipped if a cyclic dependency is detected. Signed-off-by: Francois Perrad Signed-off-by: Peter Korsgaard

graph-depends: add support for excluding host packages

2016-02-08T20:29:06+00:00

Just like the --stop-on and --exclude options allow to stop on or exclude virtual packages from the list by passing the "virtual" magic value, this commit extends the graph-depends logic to support a "host" magic value for --stop-on and --exclude. This will allow to draw the graph by stopping on host packages, or by excluding host packages. Signed-off-by: Thomas Petazzoni Tested-by: "Yann E. MORIN" Reviewed-by: "Yann E. MORIN" [Thomas: minor code beautification suggested by Yann E. Morin.] Signed-off-by: Thomas Petazzoni

graph-depends: fix handling of "virtual" in exclude_list

2016-02-08T20:28:19+00:00

The condition to determine if a virtual package should be excluded from the list due to "virtual" being passed in --exclude is under a loop iterating over each entry of the exclude_list, but it doesn't use the iterator of this list. Indeed, the condition contains: "virtual" in exclude_list which checks automatically if "virtual" was passed in the list. Due to this, there is no need for this check to be within the "for p in exclude_list" iteration. This commit fixes that by moving the check outside of the loop. Signed-off-by: Thomas Petazzoni Tested-by: "Yann E. MORIN" Acked-by: "Yann E. MORIN" Signed-off-by: Thomas Petazzoni

support/graph-depends: teach it to only check dependencies

2016-02-07T22:21:40+00:00

Add an option to graph-depends to only do the dependency checks and not generate the dot program. Signed-off-by: "Yann E. MORIN" Cc: Thomas Petazzoni Cc: Samuel Martin Signed-off-by: Thomas Petazzoni

support/graph-depends: detect circular dependencies

2016-02-07T22:20:38+00:00

Currently, if there is a circular dependency in the packages, the graph-depends script just errors out with a Python RuntimeError which is not caught, resulting in a very-long backtrace which does not provide any hint as what the real issue is (even if "RuntimeError: maximum recursion depth exceeded" is a pretty good hint at it). We fix that by recursing the dependency chain of each package, until we either end up with a package with no dependency, or with a package already seen along the current dependency chain. We need to introduce a new function, check_circular_deps(), because we can't re-use the existing ones: - remove_mandatory_deps() does not iterate, - remove_transitive_deps() does iterate, but we do not call it for the top-level package if it is not 'all' - it does not make sense to use those functions anyway, as they were not designed to _check_ but to _act_ on the dependency chain. Since we've had time-related issues in the past, we do not want to introduce yet another time-hog, so here are timings with the circular dependency check: $ time python -m cProfile -s cumtime support/scripts/graph-depends [...] 28352654 function calls (20323050 primitive calls) in 87.292 seconds Ordered by: cumulative time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.012 0.012 87.292 87.292 graph-depends:24() 21 0.000 0.000 73.685 3.509 subprocess.py:473(_eintr_retry_call) 7 0.000 0.000 73.655 10.522 subprocess.py:768(communicate) 7 73.653 10.522 73.653 10.522 {method 'read' of 'file' objects} 5/1 0.027 0.005 43.488 43.488 graph-depends:164(get_all_depends) 5 0.003 0.001 43.458 8.692 graph-depends:135(get_depends) 1 0.001 0.001 25.712 25.712 graph-depends:98(get_version) 1 0.001 0.001 13.457 13.457 graph-depends:337(remove_extra_deps) 1717 1.672 0.001 13.050 0.008 graph-depends:290(remove_transitive_deps) 9784086/2672326 5.079 0.000 11.363 0.000 graph-depends:274(is_dep) 2883343/1980154 2.650 0.000 6.942 0.000 graph-depends:262(is_dep_uncached) 1 0.000 0.000 4.529 4.529 graph-depends:121(get_targets) 2883343 1.123 0.000 1.851 0.000 graph-depends:246(is_dep_cache_insert) 9784086 1.783 0.000 1.783 0.000 graph-depends:255(is_dep_cache_lookup) 2881580 0.728 0.000 0.728 0.000 {method 'update' of 'dict' objects} 1 0.001 0.001 0.405 0.405 graph-depends:311(check_circular_deps) 12264/1717 0.290 0.000 0.404 0.000 graph-depends:312(recurse) [...] real 1m27.371s user 1m15.075s sys 0m12.673s The cumulative time spent in check_circular_deps is just below 0.5s, which is largely less than 1% of the total run time. Signed-off-by: "Yann E. MORIN" Cc: Thomas Petazzoni Cc: Samuel Martin Signed-off-by: Thomas Petazzoni

support/graph-depends: add option to specify output file

2016-02-07T22:15:30+00:00

Currently, graph-depends outputs the dotfile program to stdout, and uses stderr to trace the dependencies it is currently looking for. Redirection was done because the output was directly piped into the dot program to generate the final PDF/SVG/... dependency graph, but that meant that an error in the graph-depends script was never caught (because shell pipes only return the final command exit status, and an empty dot program is perfectly valid so dot would not complain). Add an option to tell graph-depends where to store the generated dot program, and keep stdout as the default if not specified. Signed-off-by: "Yann E. MORIN" Cc: Thomas Petazzoni Cc: Samuel Martin [Thomas: rename metavar from DOT_FILE to OUT_FILE for consistency with the rest of the new option naming.] Signed-off-by: Thomas Petazzoni