summaryrefslogtreecommitdiffstats
path: root/support/scripts/pkg-stats
Commit message (Collapse)AuthorAgeFilesLines
* support/scripts/pkg-stats: fix flake8 errorsArnout Vandecappelle (Essensium/Mind)2019-02-071-3/+5
| | | | | | | | | | | | | | | | | | | | | | Fixes the following flake8 warnings: support/scripts/pkg-stats:34:2: W605 invalid escape sequence '\$' support/scripts/pkg-stats:34:4: W605 invalid escape sequence '\(' support/scripts/pkg-stats:34:11: W605 invalid escape sequence '\$' support/scripts/pkg-stats:34:13: W605 invalid escape sequence '\(' support/scripts/pkg-stats:34:32: W605 invalid escape sequence '\)' support/scripts/pkg-stats:34:34: W605 invalid escape sequence '\)' support/scripts/pkg-stats:35:2: W605 invalid escape sequence '\s' support/scripts/pkg-stats:35:14: W605 invalid escape sequence '\S' support/scripts/pkg-stats:35:17: W605 invalid escape sequence '\s' support/scripts/pkg-stats:42:1: E302 expected 2 blank lines, found 1 support/scripts/pkg-stats:587:133: E501 line too long (157 > 132 characters) Note that the "invalid escape sequence" errors work because Python leaves the \ in place if it doesn't recognise the escape sequence. But it's better practice to use a raw string for regular expressions. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* support/scripts/pkg-stats: add latest upstream version informationThomas Petazzoni2019-02-061-0/+144
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit adds fetching the latest upstream version of each package from release-monitoring.org. The fetching process first tries to use the package mappings of the "Buildroot" distribution [1]. This mapping mechanism allows to tell release-monitoring.org what is the name of a package in a given distribution/build-system. For example, the package xutil_util-macros in Buildroot is named xorg-util-macros on release-monitoring.org. This mapping can be seen in the section "Mappings" of https://release-monitoring.org/project/15037/. If there is no mapping, then it does a regular search, and within the search results, looks for a package whose name matches the Buildroot name. Even though fetching from release-monitoring.org is a bit slow, using multiprocessing.Pool has proven to not be reliable, with some requests ending up with an exception. So we keep a serialized approach, but with a single HTTPSConnectionPool() for all queries. Long term, we hope to be able to use a database dump of release-monitoring.org instead. From an output point of view, the latest version column: - Is green when the version in Buildroot matches the latest upstream version - Is orange when the latest upstream version is unknown because the package was not found on release-monitoring.org - Is red when the version in Buildroot doesn't match the latest upstream version. Note that we are not doing anything smart here: we are just testing if the strings are equal or not. - The cell contains the link to the project on release-monitoring.org if found. - The cell indicates if the match was done using a distro mapping, or through a regular search. [1] https://release-monitoring.org/distro/Buildroot/ Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Tested-by: Matthew Weber <matthew.weber@rockwellcollins.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* support/scripts/pkg-stats: URL check using threadsMatt Weber2018-10-091-6/+19
| | | | | | | | | | | | | Adds a pool of worker threads to accelerate connection testing. ~7.5MB and 2% CPU per thread on a Intel i5-3230M CPU @ 2.60GHz. Runtime is ~3min in parallel vs ~15min. CC: Ricardo Martincoski <ricardo.martincoski@gmail.com> Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com> Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* support/scripts/pkg-stats: URL checking supportMatt Weber2018-10-091-0/+57
| | | | | | | | | | | | | | | | | | | - Adds support to check if a package has a URL and if that URL is valid by doing a header request. - Reports this information as part of the generated html output The URL data is currently gathered from the URL string provided in the Kconfig help sections for each package. This check helps ensure the URLs are valid and can be used for other scripting purposes as the product's home site/URL. CPE XML generation is an example of a case that could use this product URL as part of an automated update generation script. CC: Ricardo Martincoski <ricardo.martincoski@gmail.com> Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com> Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* support/scripts/pkg-stats: use parenthesis for printMatt Weber2018-05-191-6/+6
| | | | | | | | | Use Python 3 style print calls, in order to make pkg-stats Python 3 compliant. Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com> Reviewed-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* support/scripts/pkg-stats: replace with new Python versionThomas Petazzoni2018-04-041-394/+465
| | | | | | Rename pkg-stats-new to pkg-stats. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
* pkg-stats: add exception for nvidia-tegra23.mkThomas Petazzoni2017-07-061-0/+1
| | | | | | | This file is not a package per-se, it includes other .mk files that are packages. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* pkg-stats: take into account waf packagesThomas Petazzoni2017-07-061-0/+12
| | | | | | | The waf package infrastructure was not known by the pkg-stats script, so let's add it. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* pkg-stats: allow to sort by columnRicardo Martincoski2017-07-061-1/+12
| | | | | | | | | | | | | | | | | | | | With 2000+ packages it's not trivial to identify i.e.: - all packages that don't have a hash file; - all packages that have patches; - all packages that have code style warnings; User experience can be improved by dynamically sorting the resulting table. There is an open-source solution that does that in the client-side and requires minimal changes to our script: sorttable.js. The script is MIT licensed as stated in its website. Also add a hint to the user that the table can be sorted. Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Cc: Arnout Vandecappelle <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* tools: rename to 'utils'Thomas Petazzoni2017-07-011-1/+1
| | | | | | | | | | | After some discussion, we found out that "tools" has the four first letters identical to the "toolchain" subfolder, which makes it a bit unpractical with tab-completion. So, this commit renames "tools" to "utils", which is more tab-completion-friendly. This has been discussed with Arnout and Yann. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* tools: move check-package out of support/scripts/Yann E. MORIN2017-07-011-1/+1
| | | | | | | | | | | Move it to the top-level tools/ directory, so that it is easier to find for users. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* pkg-stats: also parse toolchain packagesThomas Petazzoni2017-04-091-2/+12
| | | | Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* pkg-stats: ignore additional package infra .mk filesThomas Petazzoni2017-04-091-0/+2
| | | | Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* pkg-stats: add new column with check-package warningsThomas Petazzoni2017-04-081-0/+9
| | | | | | | This allows the page at http://autobuild.buildroot.net/stats/ to show how many warnings returned by check-package affect each package. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* linux: add ev3dev extensionDavid Lechner2016-11-021-0/+1
| | | | | | | | This adds an ev3dev Linux drivers extension that provides Linux kernel drivers for LEGO MINDSTORMS EV3 from the ev3dev project. Signed-off-by: David Lechner <david@lechnology.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* support: pkg-stats: add efl package in packages statsRomain Naour2016-07-241-1/+0
| | | | | | | | Since efl update to 1.15 version, the efl package is a "real" Buildroot package. It doesn't contain any subdirectories anymore. Signed-off-by: Romain Naour <romain.naour@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* pkg-stats: ignore linux-ext-fbtft.mk and doc-asciidoc.mkFabio Porcedda2015-02-151-0/+2
| | | | | Signed-off-by: Fabio Porcedda <fabio.porcedda@gmail.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* pkg-stats: add kconfig and rebar infrastructuresFabio Porcedda2015-02-151-0/+32
| | | | | Signed-off-by: Fabio Porcedda <fabio.porcedda@gmail.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* Change /bin/bash shebangs into /usr/bin/env bashBjørn Forsman2014-10-251-1/+1
| | | | | | | | | | | | | | | | | Not all systems have /bin/bash (e.g. NixOS[1] doesn't). Buildroot already uses /usr/bin/env shebangs for other interpreters (perl, python), so why not bash? This changes only the shebangs used by Buildroot itself; stuff installed to the target system is left unchanged. With this applied I can run Buildroot unmodified on NixOS. [1]: http://nixos.org/ Signed-off-by: Bjørn Forsman <bjorn.forsman@gmail.com> Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* pkg-stats: add statistics about hash filesThomas Petazzoni2014-10-191-0/+26
| | | | | | | | | | | As discussed during the Buildroot meeting, this commit extends the pkg-stats script to include statistics about the number of packages having vs. not having the hash file. As of today, we have 104 packages with the hash file, and 1274 packages without. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* pkg-stats: fix whitespacesFabio Porcedda2014-05-151-5/+5
| | | | | | | | | | | Remove some spaces before tabs and add the empty line at end of file. Signed-off-by: Fabio Porcedda <fabio.porcedda@gmail.com> Cc: Jerzy Grzegorek <jerzy.grzegorek@trzebnica.net> Cc: Luca Ceresoli <luca@lucaceresoli.net> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Acked-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* pkg-stats: add luarocks perl python virtual infrastructuresFabio Porcedda2014-05-151-0/+71
| | | | | | | | | | | | Recognize the recently added infrastructures: luarocks, perl, python, virtual. Signed-off-by: Fabio Porcedda <fabio.porcedda@gmail.com> Cc: Jerzy Grzegorek <jerzy.grzegorek@trzebnica.net> Cc: Luca Ceresoli <luca@lucaceresoli.net> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Acked-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* pkg-stats: update set of files to skipJerzy Grzegorek2013-09-111-7/+10
| | | | | | | | | In BR sub-directory boot/ linux/ and package/ there are a few .mk files which aren't <package>.mk files. These files shouldn't be taken into account in package statistics. Signed-off-by: Jerzy Grzegorek <jerzy.grzegorek@trzebnica.net> Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
* package/customize: removeThomas Petazzoni2013-02-081-1/+0
| | | | | | | | | | | | This mechanism of root filesystem customization has been deprecated since a long time, so let's remove it now. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Acked-by: Luca Ceresoli <luca@lucaceresoli.net> Acked-by: "Samuel Martin" <s.martin49@gmail.com> Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
* pkg-stats: fix the broken "results" linkLuca Ceresoli2012-11-051-0/+1
| | | | | | | | At the top of the output html page there is a dangling "results" link. Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net> Acked-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
* pkg-stats: statistics about license files informationThomas Petazzoni2012-08-051-0/+25
| | | | | | | | | Since on some packages we are adding <pkg>_LICENSE but not necessarily <pkg>_LICENSE_FILES, let's add a separate statistic to track these informations. This will allow us to improve both the number of packages covered by <pkg>_LICENSE and <pkg>_LICENSE_FILES. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* pkg-stats: support <pkg>_LICENSE = lines with spacesThomas Petazzoni2012-08-051-1/+1
| | | | | | | For alignement reasons, we sometimes add spaces between <pkg>_LICENSE and the equal sign. Take this into account in pkg-stats. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* pkg-stats: ensure infratype is reset for every packageThomas Petazzoni2012-07-311-0/+1
| | | | Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* pkg-stats: add license information, rework information displayedThomas Petazzoni2012-07-311-148/+113
| | | | | | | | | | | | Now that most packages have been converted over to package infrastructures, keep only one column to show the package infrastructures. A new column, showing of the package has license information, has been added. This will help in increasing the number of packages having license metadata. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* Rename XXXTARGETS to xxx-packageArnout Vandecappelle (Essensium/Mind)2012-07-171-6/+6
| | | | | | | | With the introduction of a specific macro for host targets, it was decided to also make the names of the macros more intuitive: generic-package, autotools-package and cmake-package. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* pkg-infra: add host-xxx-package macroArnout Vandecappelle (Essensium/Mind)2012-07-171-6/+6
| | | | | | | | | | | | | | | Create host-generic-package, host-autotools-package and host-cmake-package macros. Such a macro is more intuitive to use than the $(call ...,host) construct. Also it speeds things up by having one less $(call ...) evaluation. Also includes documentation update, but not for buildroot.html. This brings the time for 'make -qp' (which is used by bash-completion) down from 1.85s to 1.35s on my laptop. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* pkg-stats: update list of packages to be skippedLuca Ceresoli2012-02-261-1/+1
| | | | | | Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net> Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
* pkg-stats: fix table layoutLuca Ceresoli2012-01-111-1/+1
| | | | | | | | The "Patch count" cell needs rowspan=2, otherwise the host/target cells are misaligned. Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net> Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
* pkg-stats: update script location in usage instructionsSven Neumann2011-10-061-1/+1
| | | | | Signed-off-by: Sven Neumann <s.neumann@raumfeld.com> Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
* pkg-stats: update list of .mk to ignoreH Hartley Sweeten2011-10-061-0/+1
| | | | | Signed-off-by: H Hartley Sweeten <hsweeten@visionengravers.com> Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
* pkg-stats: use correct variable names for convert_to_*autotoolsH Hartley Sweeten2011-10-051-1/+2
| | | | | | | | The variable convert_to_autotools is not used in the script. The correct variables are convert_to_target_autotools and convert_to_host_autotools. Signed-off-by: H Hartley Sweeten <hsweeten@visionengravers.com> Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
* pkg-stats: cnt should start with 0H Hartley Sweeten2011-10-051-1/+1
| | | | | | | | | | The package count, cnt, should start with an initial value of 0. It is incremented as each package *.mk file is checked. Starting with a value of 1 makes the first ID = 2 and results in the TOTAL being off by 1. Signed-off-by: H Hartley Sweeten <hsweeten@visionengravers.com> Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
* pkg-stats: update grep tests for package typeH Hartley Sweeten2011-10-051-6/+6
| | | | | | | | | | | | | | | | | Update the grep tests used to determine the package type. The package name and directory are now worked out magically due to: package: add helper functions to get package name and directory magically Because of this the extra arguments were removed by patches: package: remove useless arguments from GENTARGETS package: remove useless arguments from AUTOTARGETS package: remove useless arguments from CMAKETARGETS Signed-off-by: H Hartley Sweeten <hsweeten@visionengravers.com> Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
* support: move package/gnuconfig to support/gnuconfigThomas Petazzoni2011-09-171-1/+0
| | | | | | | | | | | | The CONFIG_UPDATE macro is no longer defined in package/gnuconfig/gnuconfig.mk, but instead in package/Makefile.autotools.in. It it also changed a little bit to take the directory of the package sources as argument, and the AUTOTARGETS infrastructure is updated to use this macro. [Peter: drop echo in CONFIG_UPDATE] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
* support: move scripts/ to a new support/ directoryThomas Petazzoni2011-09-171-0/+284
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
OpenPOWER on IntegriCloud