buildroot/package/python/python.mk, branch 2017.08

python: remove target Python packages from PYTHONPATH

2017-07-13T08:54:39+00:00

We currently have $(TARGET_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR)/site-packages/ inside the PYTHON_PATH variable, which gets used to define PYTHONPATH, passed to the host Python interpreter when building/installing target packages. However, this is terribly wrong, as it causes the host interpreter to potentially import target Python packages. This is wrong for several reasons: - Some Python packages might need some Python modules to be installed on the host (described in setup_requires in setup.py), but their installation currently works because by luck the corresponding Python module is installed for the target. Some of those cases were happening for real, and fixed by previous patches. - Some Python packages include some native code, therefore built for a specific CPU architecture. When you point the host Python interpreter to native libraries built for the target, you get nice build failures, such as the one affecting the python-cffi related packages. Making this change allows to fix the python-cffi related build failures: http://autobuild.buildroot.net/results/a9af84f2d845ee25e2b7d8b92aef485112b46060/ (python-cryptography) http://autobuild.buildroot.net/results/b017c4f6b4d45c0afbf06a80dbd3f2ebe5d49d20/ (python-pynacl) http://autobuild.buildroot.net/results/25144ea191ad46d851b31d3a2f0ef939f215494b/ (python-smbus-cffi) This change has been verified with the following defconfig that enables a lot of Python packages: BR2_arm=y BR2_TOOLCHAIN_EXTERNAL=y BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-arm-full-2017.05-834-gb595627.tar.bz2" BR2_TOOLCHAIN_EXTERNAL_GCC_4_9=y BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_10=y BR2_TOOLCHAIN_EXTERNAL_LOCALE=y BR2_TOOLCHAIN_EXTERNAL_CXX=y BR2_INIT_NONE=y BR2_SYSTEM_BIN_SH_NONE=y BR2_PACKAGE_PYTHON=y BR2_PACKAGE_PYTHON_ALSAAUDIO=y BR2_PACKAGE_PYTHON_ARROW=y BR2_PACKAGE_PYTHON_ATTRS=y BR2_PACKAGE_PYTHON_AUTOBAHN=y BR2_PACKAGE_PYTHON_BITSTRING=y BR2_PACKAGE_PYTHON_BOTTLE=y BR2_PACKAGE_PYTHON_CAN=y BR2_PACKAGE_PYTHON_CBOR=y BR2_PACKAGE_PYTHON_CHARDET=y BR2_PACKAGE_PYTHON_CHEETAH=y BR2_PACKAGE_PYTHON_CHERRYPY=y BR2_PACKAGE_PYTHON_CONFIGOBJ=y BR2_PACKAGE_PYTHON_CONFIGSHELL_FB=y BR2_PACKAGE_PYTHON_CRC16=y BR2_PACKAGE_PYTHON_CRCMOD=y BR2_PACKAGE_PYTHON_CSSSELECT=y BR2_PACKAGE_PYTHON_CSSUTILS=y BR2_PACKAGE_PYTHON_DAEMON=y BR2_PACKAGE_PYTHON_DIALOG=y BR2_PACKAGE_PYTHON_DICTTOXML=y BR2_PACKAGE_PYTHON_DJANGO=y BR2_PACKAGE_PYTHON_DOCOPT=y BR2_PACKAGE_PYTHON_DPKT=y BR2_PACKAGE_PYTHON_ECDSA=y BR2_PACKAGE_PYTHON_ENUM=y BR2_PACKAGE_PYTHON_FLASK_BABEL=y BR2_PACKAGE_PYTHON_FLASK_JSONRPC=y BR2_PACKAGE_PYTHON_FLASK_LOGIN=y BR2_PACKAGE_PYTHON_FLUP=y BR2_PACKAGE_PYTHON_GOBJECT=y BR2_PACKAGE_PYTHON_GUNICORN=y BR2_PACKAGE_PYTHON_HTML5LIB=y BR2_PACKAGE_PYTHON_HTTPLIB2=y BR2_PACKAGE_PYTHON_HUMANIZE=y BR2_PACKAGE_PYTHON_ID3=y BR2_PACKAGE_PYTHON_INIPARSE=y BR2_PACKAGE_PYTHON_IOWAIT=y BR2_PACKAGE_PYTHON_IPADDR=y BR2_PACKAGE_PYTHON_IPY=y BR2_PACKAGE_PYTHON_IPYTHON=y BR2_PACKAGE_PYTHON_JSON_SCHEMA_VALIDATOR=y BR2_PACKAGE_PYTHON_KEYRING=y BR2_PACKAGE_PYTHON_LIBCONFIG=y BR2_PACKAGE_PYTHON_LMDB=y BR2_PACKAGE_PYTHON_LXML=y BR2_PACKAGE_PYTHON_MAD=y BR2_PACKAGE_PYTHON_MARKDOWN=y BR2_PACKAGE_PYTHON_MELD3=y BR2_PACKAGE_PYTHON_MISTUNE=y BR2_PACKAGE_PYTHON_MSGPACK=y BR2_PACKAGE_PYTHON_MUTAGEN=y BR2_PACKAGE_PYTHON_MWSCRAPE=y BR2_PACKAGE_PYTHON_NETADDR=y BR2_PACKAGE_PYTHON_NETIFACES=y BR2_PACKAGE_PYTHON_NFC=y BR2_PACKAGE_PYTHON_NUMPY=y BR2_PACKAGE_PYTHON_PAHO_MQTT=y BR2_PACKAGE_PYTHON_PAM=y BR2_PACKAGE_PYTHON_PARAMIKO=y BR2_PACKAGE_PYTHON_PILLOW=y BR2_PACKAGE_PYTHON_POSIX_IPC=y BR2_PACKAGE_PYTHON_PSUTIL=y BR2_PACKAGE_PYTHON_PUDB=y BR2_PACKAGE_PYTHON_PYCLI=y BR2_PACKAGE_PYTHON_PYCPARSER=y BR2_PACKAGE_PYTHON_PYELFTOOLS=y BR2_PACKAGE_PYTHON_PYFTPDLIB=y BR2_PACKAGE_PYTHON_PYGAME=y BR2_PACKAGE_PYTHON_PYGAME_IMAGE=y BR2_PACKAGE_PYTHON_PYGAME_EXAMPLES=y BR2_PACKAGE_PYTHON_PYGAME_FONT=y BR2_PACKAGE_PYTHON_PYGAME_MIXER=y BR2_PACKAGE_PYTHON_PYINOTIFY=y BR2_PACKAGE_PYTHON_PYLIBFTDI=y BR2_PACKAGE_PYTHON_PYMYSQL=y BR2_PACKAGE_PYTHON_PYPARTED=y BR2_PACKAGE_PYTHON_PYPCAP=y BR2_PACKAGE_PYTHON_PYQRCODE=y BR2_PACKAGE_PYTHON_PYRATEMP=y BR2_PACKAGE_PYTHON_PYRO=y BR2_PACKAGE_PYTHON_PYROUTE2=y BR2_PACKAGE_PYTHON_PYSENDFILE=y BR2_PACKAGE_PYTHON_PYSMB=y BR2_PACKAGE_PYTHON_PYSNMP_APPS=y BR2_PACKAGE_PYTHON_PYSNMP_MIBS=y BR2_PACKAGE_PYTHON_PYSOCKS=y BR2_PACKAGE_PYTHON_PYTABLEWRITER=y BR2_PACKAGE_PYTHON_PYTRIE=y BR2_PACKAGE_PYTHON_PYUSB=y BR2_PACKAGE_PYTHON_PYXB=y BR2_PACKAGE_PYTHON_PYZMQ=y BR2_PACKAGE_PYTHON_REQUESTS_TOOLBELT=y BR2_PACKAGE_PYTHON_RPI_GPIO=y BR2_PACKAGE_PYTHON_RTSLIB_FB=y BR2_PACKAGE_PYTHON_SDNOTIFY=y BR2_PACKAGE_PYTHON_SERIAL=y BR2_PACKAGE_PYTHON_SETPROCTITLE=y BR2_PACKAGE_PYTHON_SH=y BR2_PACKAGE_PYTHON_SHUTILWHICH=y BR2_PACKAGE_PYTHON_SIMPLEJSON=y BR2_PACKAGE_PYTHON_SMBUS_CFFI=y BR2_PACKAGE_PYTHON_SOCKETIO=y BR2_PACKAGE_PYTHON_SORTEDCONTAINERS=y BR2_PACKAGE_PYTHON_SPIDEV=y BR2_PACKAGE_PYTHON_THRIFT=y BR2_PACKAGE_PYTHON_TOMAKO=y BR2_PACKAGE_PYTHON_TREQ=y BR2_PACKAGE_PYTHON_U_MSGPACK=y BR2_PACKAGE_PYTHON_UBJSON=y BR2_PACKAGE_PYTHON_UJSON=y BR2_PACKAGE_PYTHON_URLLIB3=y BR2_PACKAGE_PYTHON_VERSIONTOOLS=y BR2_PACKAGE_PYTHON_WATCHDOG=y BR2_PACKAGE_PYTHON_WEB2PY=y BR2_PACKAGE_PYTHON_WEBPY=y BR2_PACKAGE_PYTHON_WHOOSH=y BR2_PACKAGE_PYTHON_WS4PY=y BR2_PACKAGE_PYTHON_WSACCEL=y BR2_PACKAGE_PYTHON_XLUTILS=y Signed-off-by: Thomas Petazzoni Reviewed-by: Yegor Yefremov Signed-off-by: Thomas Petazzoni

Globally replace $(HOST_DIR)/usr/bin with $(HOST_DIR)/bin

2017-07-05T13:19:29+00:00

Since things are no longer installed in $(HOST_DIR)/usr, the callers should also not refer to it. This is a mechanical change with git grep -l '$(HOST_DIR)/usr/bin' | xargs sed -i 's%$(HOST_DIR)/usr/bin%$(HOST_DIR)/bin%g' Signed-off-by: Arnout Vandecappelle (Essensium/Mind) Signed-off-by: Thomas Petazzoni

python: use the new gettext logic

2017-07-04T23:34:03+00:00

This commit switches to use the new gettext logic, which involves using TARGET_NLS_DEPENDENCIES instead of hand-encoded dependencies on gettext/host-gettext. Signed-off-by: Thomas Petazzoni

package makefiles: clean up backslash spacing.

2017-04-22T13:57:23+00:00

The check-package script when ran gave warnings on only using one space before backslashes on all of these makefiles. This patch cleans up all warnings related to the one space before backslashes rule in the make files in the package directory. Signed-off-by: Adam Duskett Signed-off-by: Thomas Petazzoni

python: use SPDX short identifier for license string

2017-04-09T11:47:54+00:00

Signed-off-by: Rahul Bedarkar Signed-off-by: Thomas Petazzoni

python2: remove full path from .pyc

2017-04-01T16:30:07+00:00

.pyc files include path to source .py file. This patch changes the way `pycompile.py' is launched in order to only keep the part relative to $TARGET_DIR. This work was sponsored by `BA Robotic Systems'. Signed-off-by: Jérôme Pouiller Reviewed-by: Arnout Vandecappelle (Essensium/Mind) Reviewed-by: Samuel Martin Signed-off-by: Thomas Petazzoni

arch, linux, package: remove whitespaces

2017-03-29T21:30:37+00:00

Whitespaces were searched using the following regex: [ ]{1,}\t and then manually removed in most of the cases. For xserver_xorg-server.mk, tabs before backslashes were removed. Signed-off-by: Bernd Kuhls Signed-off-by: Thomas Petazzoni

python2: generate reproducible .pyc

2017-03-20T22:03:15+00:00

.pyc files contain the modification time of the corresponding .py source. In order to make the build reproducible, we fix the modification time of all .py before compiling .pyc files. In addition, since pycompile relies on the modification time to know if a file needs to be recompiled, it is safer to force recompilation of all source files. This work was sponsored by `BA Robotic Systems'. Signed-off-by: Jérôme Pouiller Reviewed-by: Samuel Martin [Thomas: do not register PYTHON_FIX_TIME as a PYTHON_TARGET_FINALIZE_HOOKS, instead call it inside PYTHON_CREATE_PYC_FILES before doing the byte compilation.] Signed-off-by: Thomas Petazzoni

package/python: add optional support for libintl

2017-03-12T15:07:43+00:00

Python links to gettext when available: $ output/host/usr/bin/i586-buildroot-linux-uclibc-readelf -a output/target/usr/lib/python2.7/lib-dynload/_locale.so | grep NEEDED 0x00000001 (NEEDED) Shared library: [libintl.so.8] 0x00000001 (NEEDED) Shared library: [libpython2.7.so.1.0] 0x00000001 (NEEDED) Shared library: [libc.so.0] In comparison the same library compiled without gettext: $ output/host/usr/bin/i586-buildroot-linux-uclibc-readelf -a output/target/usr/lib/python2.7/lib-dynload/_locale.so | grep NEEDED 0x00000001 (NEEDED) Shared library: [libpython2.7.so.1.0] 0x00000001 (NEEDED) Shared library: [libc.so.0] Signed-off-by: Bernd Kuhls Signed-off-by: Thomas Petazzoni

python: make hashlib and readline modules really optional

2017-03-09T21:35:23+00:00

The BR2_PACKAGE_PYTHON_{READLINE,HASHLIB} options were so far only bringing in the necessary dependencies, relying on the Python build system to automatically detect them. However, this means that even if one of those option was disabled, if their dependency was found, Python would build the corresponding module, which is really not what the user would expect. For example, if you have: BR2_PACKAGE_READLINE=y # BR2_PACKAGE_PYTHON_READLINE is not set Then you would still get the readline Python module built and installed. This commit fixes that by adding new --{enable,disable} options, and use them in python.mk. Signed-off-by: Thomas Petazzoni Reviewed-by: Yegor Yefremov Signed-off-by: Thomas Petazzoni