python3: bump to 3.5.1

The major changes in terms of Buildroot packaging are:

 - Due to PEP488, Python no longer generates .pyc (unoptimized) and
   .pyo (optimized) byte-code files. Instead, it generates <foo>.pyc,
   <foo>.opt-1.pyc and <foo>.opt-2.pyc. Therefore, we removed the
   --disable-pyo-build option and kept only the --disable-pyc-build
   option, which completely disables building all .pyc files. In
   addition, since the optimized .opt-X.pyc files don't work if the
   corresponding un-optimized .pyc file is not present, we are for the
   moment unconditionally removing the optimized ones (keeping both
   the unoptimized and optimized ones doubles the required filesystem
   size!). So basically we preserve the behavior we had before this
   commit:

     BR2_PACKAGE_PYTHON3_PY_ONLY -> only *.py
     BR2_PACKAGE_PYTHON3_PYC_ONLY -> only non-optimized *.pyc
     BR2_PACKAGE_PYTHON3_PY_PYC -> both the *.py and non-optimized *.pyc

   To achieve this, the TARGET_FINALIZE_HOOKS are reworked:

    PYTHON3_REMOVE_PY_FILES is responsible for removing *.py files in
    the BR2_PACKAGE_PYTHON3_PYC_ONLY case.

    PYTHON3_REMOVE_PYC_FILES is responsible for removing *.pyc files
    in the BR2_PACKAGE_PYTHON3_PY_ONLY case.

    PYTHON3_REMOVE_OPTIMIZED_PYC_FILES is responsible for removing the
    optimized *.opt-1.pyc and *.opt-2.pyc files, which is done
    unconditionally.

 - The PEP3147 disabling patch had to be significantly reworked due to
   the code having changed heavily. The code was moved into a
   _bootstrap_external.py, which is a "frozen" Python module, i.e a
   module generated into a .h file at compile time using the
   _freeze_importlib program.

 - Due to the above, we now need to regenerate importlib.h at build
   time. Unfortunately, for the target Python _freeze_importlib is
   built for the target, so we can't run it on the build machine. To
   fix this, we copy the _freeze_importlib program from the
   host-python in $(HOST_DIR), and then patch the target python to use
   it. Since the same solution can be used for 'pgen', we do it, and
   avoid having to touch the graminit.{c,h} files.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

1 files changed, 43 insertions, 26 deletions

diff --git a/package/python3/0016-Add-importlib-fix-for-PEP-3147-issue.patch b/package/python3/0016-Add-importlib-fix-for-PEP-3147-issue.patch
index 02f1d164f7..54a0b87ada 100644
--- a/package/python3/0016-Add-importlib-fix-for-PEP-3147-issue.patch
+++ b/package/python3/0016-Add-importlib-fix-for-PEP-3147-issue.patch
@@ -1,4 +1,4 @@
-From 1153f503a38daf6388021575f1ad6ce8b702911b Mon Sep 17 00:00:00 2001
+From d2b5be9a0627e38d2280ef865dab3e74b7d2bde4 Mon Sep 17 00:00:00 2001
 From: Christophe Vu-Brugier <cvubrugier@fastmail.fm>
 Date: Wed, 23 Dec 2015 11:46:14 +0100
 Subject: [PATCH] Add importlib fix for PEP 3147 issue
@@ -19,44 +19,51 @@ This effectively disables PEP 3147 for:
 
 Signed-off-by: Christophe Vu-Brugier <cvubrugier@fastmail.fm>
 ---
- Lib/importlib/_bootstrap.py | 26 +++++---------------------
- 1 file changed, 5 insertions(+), 21 deletions(-)
+ Lib/importlib/_bootstrap_external.py | 37 +++++-------------------------------
+ 1 file changed, 5 insertions(+), 32 deletions(-)
 
-diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
-index 5b91c05..a1755ec 100644
---- a/Lib/importlib/_bootstrap.py
-+++ b/Lib/importlib/_bootstrap.py
-@@ -444,8 +444,6 @@ def cache_from_source(path, debug_override=None):
-     If debug_override is not None, then it must be a boolean and is used in
-     place of sys.flags.optimize.
+diff --git a/Lib/importlib/_bootstrap_external.py b/Lib/importlib/_bootstrap_external.py
+index 616b17f..e30129c 100644
+--- a/Lib/importlib/_bootstrap_external.py
++++ b/Lib/importlib/_bootstrap_external.py
+@@ -255,8 +255,6 @@ def cache_from_source(path, debug_override=None, *, optimization=None):
+     a True value is the same as setting 'optimization' to the empty string
+     while a False value is equivalent to setting 'optimization' to '1'.
  
 -    If sys.implementation.cache_tag is None then NotImplementedError is raised.
 -
      """
-     debug = not sys.flags.optimize if debug_override is None else debug_override
-     if debug:
-@@ -454,33 +452,19 @@ def cache_from_source(path, debug_override=None):
-         suffixes = OPTIMIZED_BYTECODE_SUFFIXES
+     if debug_override is not None:
+         _warnings.warn('the debug_override parameter is deprecated; use '
+@@ -267,10 +265,7 @@ def cache_from_source(path, debug_override=None, *, optimization=None):
+         optimization = '' if debug_override else 1
      head, tail = _path_split(path)
      base, sep, rest = tail.rpartition('.')
 -    tag = sys.implementation.cache_tag
 -    if tag is None:
 -        raise NotImplementedError('sys.implementation.cache_tag is None')
--    filename = ''.join([(base if base else rest), sep, tag, suffixes[0]])
--    return _path_join(head, _PYCACHE, filename)
-+    filename = ''.join([(base if base else rest), suffixes[0]])
-+    return _path_join(head, filename)
+-    almost_filename = ''.join([(base if base else rest), sep, tag])
++    almost_filename = ''.join([(base if base else rest)])
+     if optimization is None:
+         if sys.flags.optimize == 0:
+             optimization = ''
+@@ -281,39 +276,17 @@ def cache_from_source(path, debug_override=None, *, optimization=None):
+         if not optimization.isalnum():
+             raise ValueError('{!r} is not alphanumeric'.format(optimization))
+         almost_filename = '{}.{}{}'.format(almost_filename, _OPT, optimization)
+-    return _path_join(head, _PYCACHE, almost_filename + BYTECODE_SUFFIXES[0])
++    return _path_join(head, almost_filename + BYTECODE_SUFFIXES[0])
  
  
  def source_from_cache(path):
-     """Given the path to a .pyc./.pyo file, return the path to its .py file.
+     """Given the path to a .pyc. file, return the path to its .py file.
  
-     The .pyc/.pyo file does not need to exist; this simply returns the path to
--    the .py file calculated to correspond to the .pyc/.pyo file.  If path does
--    not conform to PEP 3147 format, ValueError will be raised. If
+     The .pyc file does not need to exist; this simply returns the path to
+-    the .py file calculated to correspond to the .pyc file.  If path does
+-    not conform to PEP 3147/488 format, ValueError will be raised. If
 -    sys.implementation.cache_tag is None then NotImplementedError is raised.
-+    the .py file calculated to correspond to the .pyc/.pyo file.
- 
+-
++    the .py file calculated to correspond to the .pyc file.
      """
 -    if sys.implementation.cache_tag is None:
 -        raise NotImplementedError('sys.implementation.cache_tag is None')
@@ -65,9 +72,19 @@ index 5b91c05..a1755ec 100644
 -    if pycache != _PYCACHE:
 -        raise ValueError('{} not bottom-level directory in '
 -                         '{!r}'.format(_PYCACHE, path))
--    if pycache_filename.count('.') != 2:
--        raise ValueError('expected only 2 dots in '
+-    dot_count = pycache_filename.count('.')
+-    if dot_count not in {2, 3}:
+-        raise ValueError('expected only 2 or 3 dots in '
 -                         '{!r}'.format(pycache_filename))
+-    elif dot_count == 3:
+-        optimization = pycache_filename.rsplit('.', 2)[-2]
+-        if not optimization.startswith(_OPT):
+-            raise ValueError("optimization portion of filename does not start "
+-                             "with {!r}".format(_OPT))
+-        opt_level = optimization[len(_OPT):]
+-        if not opt_level.isalnum():
+-            raise ValueError("optimization level {!r} is not an alphanumeric "
+-                             "value".format(optimization))
 -    base_filename = pycache_filename.partition('.')[0]
 +    head, filename = _path_split(path)
 +    base_filename = filename.partition('.')[0]