Merge pull request #2666 from kivy/release-2022.09.04

* Add option for copying arbitratry xml files to src/main/res/xml without touching manifest

* handle the case of getting list as res_xml

* Remove stray - in output file name

The final component in the package name is just the extension, so
prefixing it with a - means the output file ends in -.<ext>.

* Resolve absolute path to local recipes (#2640)

This is necessary when using patches in a local recipe since
`Recipe.apply_patch` assumes the recipe directory is absolute and uses
`patch -d` to change directories. It could just be fixed there, but this
ensures that recipe directories are always absolute.

Closes: #2623

* Fix webview Back button behaviour (#2636)

* Fixes TypeError: str.join() takes exactly one argument (2 given) in hostpython3/__init__.py", line 69 (#2642)

* Update __init__.py

.buildozer/android/platform/python-for-android/pythonforandroid/recipes/hostpython3/__init__.py", line 69, in get_recipe_env
    env["PKG_CONFIG_PATH"] = os.pathsep.join(
TypeError: str.join() takes exactly one argument (2 given)

* according of 

#2642 (review)

* RTSP support for ffmpeg

Added UDP and TCP protocols to "--enable-procol" to supporting RTSP streams over each.

Added RTSP to "--enable-demuxer" to supporting RTSP with ffmpeg.

* Fixes some E275 - assert is a keyword. (#2647)

* Add icon-bg and icon-fg to fix_args + adds a missing folder in webview dir template (#2633)

* Add icon-bg and icon-fg to fix_args

Fix the icon-bg and icon-fg arguments and also create the folder
mipmap-anydpi-v26 if it doesn't exists before trying to create a file
inside.

* Use existing ensure_dir function to create anydpi_dir

* Add mipmap-anydpi-v26 to bootstrap template

This makes available the directory in the final template so it's not
needed to check the directory exists.

* Updates matplotlib, fixes an issue related to shared libc++ (#2645)

* Updates matplotlib, fixes an issue related to shared libc++

* Cleanup

* Update supported Python versions (#2656)

* Remove six and enum34 dependency (#2657)

* Force --platform=linux/amd64 in Dockerfile (#2660)

* Use p4a_install instead of install, as a file named INSTALL is already present. (#2663)

* Update CHANGELOG.md & change version for release 2022.09.04

Co-authored-by: Eero af Heurlin <[email protected]>
Co-authored-by: Dan Nicholson <[email protected]>
Co-authored-by: Andre Miras <[email protected]>
Co-authored-by: Andy Trofimov <[email protected]>
Co-authored-by: Akshay Arora <[email protected]>
Co-authored-by: --=FurtiF™=-- <[email protected]>
Co-authored-by: alickc <[email protected]>
Co-authored-by: danigm <[email protected]>

Author: 9 people

.github/workflows/push.yml

@@ -3,9 +3,9 @@ name: Unit tests & build apps
 on: ['push', 'pull_request']
 
 env:
-  APK_ARTIFACT_FILENAME: bdist_unit_tests_app-debug-1.1-.apk
-  AAB_ARTIFACT_FILENAME: bdist_unit_tests_app-release-1.1-.aab
-  AAR_ARTIFACT_FILENAME: bdist_unit_tests_app-release-1.1-.aar
+  APK_ARTIFACT_FILENAME: bdist_unit_tests_app-debug-1.1.apk
+  AAB_ARTIFACT_FILENAME: bdist_unit_tests_app-release-1.1.aab
+  AAR_ARTIFACT_FILENAME: bdist_unit_tests_app-release-1.1.aar
   PYTHONFORANDROID_PREREQUISITES_INSTALL_INTERACTIVE: 0
 
 jobs:

CHANGELOG.md

@@ -1,5 +1,48 @@
 # Changelog
 
+## [v2022.09.04](https://github.com/kivy/python-for-android/tree/v2022.09.04) (2022-09-04)
+
+[Full Changelog](https://github.com/kivy/python-for-android/compare/v2022.07.20...v2022.09.04)
+
+**Fixed bugs:**
+
+- Matplotlib failed to import properly on an APK from Buildozer and Kivy [\#2643](https://github.com/kivy/python-for-android/issues/2643)
+
+**Closed issues:**
+
+- KeyError: Matplotlib with kivy android [\#2658](https://github.com/kivy/python-for-android/issues/2658)
+- KeyError: Matplotlib [\#2659](https://github.com/kivy/python-for-android/issues/2659)
+- Upgrade from NDK 19b to 23b causes problems with Pandas library [\#2654](https://github.com/kivy/python-for-android/issues/2654)
+- Update Dockerfile for ARM [\#2653](https://github.com/kivy/python-for-android/issues/2653)
+- Apple M2 chip doesn't generate apk: compiling error on liblzma [\#2652](https://github.com/kivy/python-for-android/issues/2652)
+- aiohttp/\_http\_parser.pyx:46:0: '\_headers.pxi' not found [\#2651](https://github.com/kivy/python-for-android/issues/2651)
+- \[Question\] Pip SSL ? [\#2649](https://github.com/kivy/python-for-android/issues/2649)
+- Colab gives me as error "No module named 'typing\_extensions' ", even if before with the same compilation it worked [\#2648](https://github.com/kivy/python-for-android/issues/2648)
+- \[Question\] Java Files [\#2646](https://github.com/kivy/python-for-android/issues/2646)
+- Using foreground services will cause wired behaviour on Android 8 [\#2641](https://github.com/kivy/python-for-android/issues/2641)
+- Can't apply patches with relative paths for local recipe [\#2623](https://github.com/kivy/python-for-android/issues/2623)
+- Compile for x86 on MacOS [\#2215](https://github.com/kivy/python-for-android/issues/2215)
+- splash always loading  [\#1907](https://github.com/kivy/python-for-android/issues/1907)
+- python-for-android.readthedocs.io has problems updating, apparently [\#1709](https://github.com/kivy/python-for-android/issues/1709)
+- Webview apps not working on Android [\#1644](https://github.com/kivy/python-for-android/issues/1644)
+
+**Merged pull requests:**
+
+- `liblzma`: Use `p4a_install` instead of `install`, as a file named `INSTALL` is already present. [\#2663](https://github.com/kivy/python-for-android/pull/2663) ([misl6](https://github.com/misl6))
+- Force `--platform=linux/amd64` in Dockerfile [\#2660](https://github.com/kivy/python-for-android/pull/2660) ([misl6](https://github.com/misl6))
+- Remove six and enum34 dependency [\#2657](https://github.com/kivy/python-for-android/pull/2657) ([misl6](https://github.com/misl6))
+- Update supported Python versions [\#2656](https://github.com/kivy/python-for-android/pull/2656) ([misl6](https://github.com/misl6))
+- Fixes some E275 - assert is a keyword. [\#2647](https://github.com/kivy/python-for-android/pull/2647) ([misl6](https://github.com/misl6))
+- Updates matplotlib, fixes an issue related to shared libc++ [\#2645](https://github.com/kivy/python-for-android/pull/2645) ([misl6](https://github.com/misl6))
+- RTSP support for ffmpeg [\#2644](https://github.com/kivy/python-for-android/pull/2644) ([alicakici1234](https://github.com/alicakici1234))
+- Fixes TypeError: str.join\(\) takes exactly one argument \(2 given\) in hostpython3/\_\_init\_\_.py", line 69 [\#2642](https://github.com/kivy/python-for-android/pull/2642) ([Furtif](https://github.com/Furtif))
+- Resolve absolute path to local recipes [\#2640](https://github.com/kivy/python-for-android/pull/2640) ([dbnicholson](https://github.com/dbnicholson))
+- Merges master into develop after release 2022.07.20 [\#2639](https://github.com/kivy/python-for-android/pull/2639) ([misl6](https://github.com/misl6))
+- Fix webview Back button behaviour [\#2636](https://github.com/kivy/python-for-android/pull/2636) ([interlark](https://github.com/interlark))
+- Add icon-bg and icon-fg to fix\_args [\#2633](https://github.com/kivy/python-for-android/pull/2633) ([danigm](https://github.com/danigm))
+- Remove stray - in output file name [\#2581](https://github.com/kivy/python-for-android/pull/2581) ([dbnicholson](https://github.com/dbnicholson))
+- Add option for adding files to res/xml without touching manifest [\#2330](https://github.com/kivy/python-for-android/pull/2330) ([rambo](https://github.com/rambo))
+
 ## [v2022.07.20](https://github.com/kivy/python-for-android/tree/v2022.07.20) (2022-07-20)
 
 [Full Changelog](https://github.com/kivy/python-for-android/compare/v2022.03.13...v2022.07.20)

Dockerfile

@@ -15,7 +15,10 @@
 #     Use 'docker run' without '--rm' flag for keeping the container and use
 #     'docker commit <container hash> <new image>' to extend the original image
 
-FROM ubuntu:20.04
+# If platform is not specified, by default the target platform of the build request is used.
+# This is not what we want, as Google doesn't provide a linux/arm64 compatible NDK.
+# See: https://docs.docker.com/engine/reference/builder/#from
+FROM --platform=linux/amd64 ubuntu:20.04
 
 # configure locale
 RUN apt -y update -qq > /dev/null \

Makefile

@@ -106,17 +106,17 @@ docker/run/command: docker/build
 
 docker/run/make/with-artifact/apk/%: docker/build
 	docker run --name p4a-latest --env-file=.env $(DOCKER_IMAGE) make $*
-	docker cp p4a-latest:/home/user/app/testapps/on_device_unit_tests/bdist_unit_tests_app-debug-1.1-.apk ./apks
+	docker cp p4a-latest:/home/user/app/testapps/on_device_unit_tests/bdist_unit_tests_app-debug-1.1.apk ./apks
 	docker rm -fv p4a-latest
 
 docker/run/make/with-artifact/aar/%: docker/build
 	docker run --name p4a-latest --env-file=.env $(DOCKER_IMAGE) make $*
-	docker cp p4a-latest:/home/user/app/testapps/on_device_unit_tests/bdist_unit_tests_app-release-1.1-.aar ./aars
+	docker cp p4a-latest:/home/user/app/testapps/on_device_unit_tests/bdist_unit_tests_app-release-1.1.aar ./aars
 	docker rm -fv p4a-latest
 
 docker/run/make/with-artifact/aab/%: docker/build
 	docker run --name p4a-latest --env-file=.env $(DOCKER_IMAGE) make $*
-	docker cp p4a-latest:/home/user/app/testapps/on_device_unit_tests/bdist_unit_tests_app-release-1.1-.aab ./aabs
+	docker cp p4a-latest:/home/user/app/testapps/on_device_unit_tests/bdist_unit_tests_app-release-1.1.aab ./aabs
 	docker rm -fv p4a-latest
 
 docker/run/make/rebuild_updated_recipes: docker/build

pythonforandroid/__init__.py

@@ -1 +1 @@
-__version__ = '2022.07.20'
+__version__ = '2022.09.04'

pythonforandroid/bootstraps/common/build/build.py

@@ -508,12 +508,20 @@ def make_package(args):
     url_scheme = 'kivy'
 
     # Copy backup rules file if specified and update the argument
+    res_xml_dir = join(res_dir, 'xml')
     if args.backup_rules:
-        res_xml_dir = join(res_dir, 'xml')
         ensure_dir(res_xml_dir)
         shutil.copy(join(args.private, args.backup_rules), res_xml_dir)
         args.backup_rules = split(args.backup_rules)[1][:-4]
 
+    # Copy res_xml files to src/main/res/xml
+    if args.res_xmls:
+        ensure_dir(res_xml_dir)
+        for xmlpath in args.res_xmls:
+            if not os.path.exists(xmlpath):
+                xmlpath = join(args.private, xmlpath)
+            shutil.copy(xmlpath, res_xml_dir)
+
     # Render out android manifest:
     manifest_path = "src/main/AndroidManifest.xml"
     render_args = {
@@ -803,6 +811,8 @@ def parse_args_and_make_package(args=None):
                           'filename containing xml. The filename should be '
                           'located relative to the python-for-android '
                           'directory'))
+    ap.add_argument('--res_xml', dest='res_xmls', action='append', default=[],
+                    help='Add files to res/xml directory (for example device-filters)', nargs='+')
     ap.add_argument('--with-billing', dest='billing_pubkey',
                     help='If set, the billing service will be added (not implemented)')
     ap.add_argument('--add-source', dest='extra_source_dirs', action='append',
@@ -901,6 +911,9 @@ def _read_configuration():
     if args.permissions and isinstance(args.permissions[0], list):
         args.permissions = [p for perm in args.permissions for p in perm]
 
+    if args.res_xmls and isinstance(args.res_xmls[0], list):
+        args.res_xmls = [x for res in args.res_xmls for x in res]
+
     if args.try_system_python_compile:
         # Hardcoding python2.7 is okay for now, as python3 skips the
         # compilation anyway

pythonforandroid/bootstraps/service_only/build/src/main/java/org/kivy/android/PythonActivity.java

@@ -177,19 +177,22 @@ public void loadLibraries() {
             new File(getApplicationInfo().nativeLibraryDir));
     }
 
-    long lastBackClick = SystemClock.elapsedRealtime();
+    long lastBackClick = 0;
     @Override
     public boolean onKeyDown(int keyCode, KeyEvent event) {
-        // If it wasn't the Back key or there's no web page history, bubble up to the default
-        // system behavior (probably exit the activity)
-        if (SystemClock.elapsedRealtime() - lastBackClick > 2000){
+        // Check if the key event was the Back button
+        if (keyCode == KeyEvent.KEYCODE_BACK) {
+            // If there's no web page history, bubble up to the default
+            // system behavior (probably exit the activity)
+            if (SystemClock.elapsedRealtime() - lastBackClick > 2000){
+                lastBackClick = SystemClock.elapsedRealtime();
+                Toast.makeText(this, "Tap again to close the app", Toast.LENGTH_LONG).show();
+                return true;
+            }
+
             lastBackClick = SystemClock.elapsedRealtime();
-            Toast.makeText(this, "Click again to close the app",
-            Toast.LENGTH_LONG).show();
-            return true;
         }
 
-        lastBackClick = SystemClock.elapsedRealtime();
         return super.onKeyDown(keyCode, event);
     }
 

pythonforandroid/bootstraps/webview/build/src/main/java/org/kivy/android/PythonActivity.java

@@ -32,6 +32,7 @@
 import android.widget.AbsoluteLayout;
 import android.view.ViewGroup.LayoutParams;
 
+import android.webkit.WebBackForwardList;
 import android.webkit.WebViewClient;
 import android.webkit.WebView;
 import android.webkit.CookieManager;
@@ -269,24 +270,30 @@ public static ViewGroup getLayout() {
         return   mLayout;
     }
 
-    long lastBackClick = SystemClock.elapsedRealtime();
+    long lastBackClick = 0;
     @Override
     public boolean onKeyDown(int keyCode, KeyEvent event) {
-        // Check if the key event was the Back button and if there's history
-        if ((keyCode == KeyEvent.KEYCODE_BACK) && mWebView.canGoBack()) {
-            mWebView.goBack();
-            return true;
-        }
-        // If it wasn't the Back key or there's no web page history, bubble up to the default
-        // system behavior (probably exit the activity)
-        if (SystemClock.elapsedRealtime() - lastBackClick > 2000){
+        // Check if the key event was the Back button
+        if (keyCode == KeyEvent.KEYCODE_BACK) {
+            // Go back if there is web page history behind,
+            // but not to the start preloader
+            WebBackForwardList webViewBackForwardList = mWebView.copyBackForwardList();
+            if (webViewBackForwardList.getCurrentIndex() > 1) {
+                mWebView.goBack();
+                return true;
+            }
+
+            // If there's no web page history, bubble up to the default
+            // system behavior (probably exit the activity)
+            if (SystemClock.elapsedRealtime() - lastBackClick > 2000){
+                lastBackClick = SystemClock.elapsedRealtime();
+                Toast.makeText(this, "Tap again to close the app", Toast.LENGTH_LONG).show();
+                return true;
+            }
+
             lastBackClick = SystemClock.elapsedRealtime();
-            Toast.makeText(this, "Click again to close the app",
-            Toast.LENGTH_LONG).show();
-            return true;
         }
 
-        lastBackClick = SystemClock.elapsedRealtime();
         return super.onKeyDown(keyCode, event);
     }
 

pythonforandroid/bootstraps/webview/build/src/main/res/mipmap-anydpi-v26/.gitkeep

@@ -45,7 +45,7 @@ def get_dependency_tuple_list_for_recipe(recipe, blacklist=None):
     """
     if blacklist is None:
         blacklist = set()
-    assert(type(blacklist) == set)
+    assert type(blacklist) == set
     if recipe.depends is None:
         dependencies = []
     else:

pythonforandroid/graph.py

@@ -1,7 +1,6 @@
 from os.path import basename, dirname, exists, isdir, isfile, join, realpath, split
 import glob
 from shutil import rmtree
-from six import with_metaclass
 
 import hashlib
 from re import match
@@ -40,7 +39,7 @@ def __new__(cls, name, bases, dct):
         return super().__new__(cls, name, bases, dct)
 
 
-class Recipe(with_metaclass(RecipeMeta)):
+class Recipe(metaclass=RecipeMeta):
     _url = None
     '''The address from which the recipe may be downloaded. This is not
     essential, it may be omitted if the source is available some other
@@ -123,8 +122,8 @@ class Recipe(with_metaclass(RecipeMeta)):
     """
 
     need_stl_shared = False
-    '''Some libraries or python packages may need to be linked with android's
-    stl. We can automatically do this for any recipe if we set this property to
+    '''Some libraries or python packages may need the c++_shared in APK.
+    We can automatically do this for any recipe if we set this property to
     `True`'''
 
     stl_lib_name = 'c++_shared'
@@ -492,20 +491,6 @@ def get_recipe_env(self, arch=None, with_flags_in_cc=True):
         if arch is None:
             arch = self.filtered_archs[0]
         env = arch.get_env(with_flags_in_cc=with_flags_in_cc)
-
-        if self.need_stl_shared:
-            env['CPPFLAGS'] = env.get('CPPFLAGS', '')
-            env['CPPFLAGS'] += ' -I{}'.format(self.ctx.ndk.libcxx_include_dir)
-
-            env['CXXFLAGS'] = env['CFLAGS'] + ' -frtti -fexceptions'
-
-            if with_flags_in_cc:
-                env['CXX'] += ' -frtti -fexceptions'
-
-            env['LDFLAGS'] += ' -L{}'.format(arch.ndk_lib_dir)
-            env['LIBS'] = env.get('LIBS', '') + " -l{}".format(
-                self.stl_lib_name
-            )
         return env
 
     def prebuild_arch(self, arch):

pythonforandroid/recipe.py

@@ -3,10 +3,7 @@
 
 class CppyRecipe(PythonRecipe):
     site_packages_name = 'cppy'
-
-    # Pin to commit: `Nucleic migration and project documentation`,
-    # because the official releases are too old, at time of writing
-    version = '4e0b956'
+    version = '1.1.0'
     url = 'https://github.com/nucleic/cppy/archive/{version}.zip'
     call_hostpython_via_targetpython = False
     # to be detected by the matplotlib install script

pythonforandroid/recipes/cppy/__init__.py

@@ -82,7 +82,7 @@ def build_arch(self, arch):
                     '--enable-parser=aac,ac3,h261,h264,mpegaudio,mpeg4video,mpegvideo,vc1',
                     '--enable-decoder=aac,h264,mpeg4,mpegvideo',
                     '--enable-muxer=h264,mov,mp4,mpeg2video',
-                    '--enable-demuxer=aac,h264,m4v,mov,mpegvideo,vc1',
+                    '--enable-demuxer=aac,h264,m4v,mov,mpegvideo,vc1,rtsp',
                 ]
 
             # needed to prevent _ffmpeg.so: version node not found for symbol av_init_packet@LIBAVFORMAT_52
@@ -100,7 +100,7 @@ def build_arch(self, arch):
             # other flags:
             flags += [
                 '--enable-filter=aresample,resample,crop,adelay,volume,scale',
-                '--enable-protocol=file,http,hls',
+                '--enable-protocol=file,http,hls,udp,tcp',
                 '--enable-small',
                 '--enable-hwaccels',
                 '--enable-pic',

pythonforandroid/recipes/ffmpeg/__init__.py

@@ -67,7 +67,7 @@ def get_recipe_env(self, arch=None):
         openssl_prereq = OpenSSLPrerequisite()
         if env.get("PKG_CONFIG_PATH", ""):
             env["PKG_CONFIG_PATH"] = os.pathsep.join(
-                openssl_prereq.pkg_config_location, env["PKG_CONFIG_PATH"]
+                [openssl_prereq.pkg_config_location, env["PKG_CONFIG_PATH"]]
             )
         else:
             env["PKG_CONFIG_PATH"] = openssl_prereq.pkg_config_location

pythonforandroid/recipes/hostpython3/__init__.py

@@ -3,28 +3,9 @@
 
 class KiwiSolverRecipe(CppCompiledComponentsPythonRecipe):
     site_packages_name = 'kiwisolver'
-    # Pin to commit `docs: attempt to fix doc building`, the latest one
-    # at the time of writing, just to be sure that we have te most up to date
-    # version, but it should be pinned to an official release once the c++
-    # changes that we want to include are merged to master branch
-    #   Note: the commit we want to include is
-    #         `Cppy use update and c++11 compatibility` (4858730)
-    version = '0846189'
+    version = '1.3.2'
     url = 'https://github.com/nucleic/kiwi/archive/{version}.zip'
     depends = ['cppy']
 
-    def get_recipe_env(self, arch=None, with_flags_in_cc=True):
-        env = super().get_recipe_env(arch, with_flags_in_cc)
-        if self.need_stl_shared:
-            # kiwisolver compile flags does not honor the standard flags:
-            # `CPPFLAGS` and `LDLIBS`, so we put in `CFLAGS` and `LDFLAGS` to
-            # correctly link with the `c++_shared` library
-            env['CFLAGS'] += f' -I{self.ctx.ndk.libcxx_include_dir}'
-            env['CFLAGS'] += ' -frtti -fexceptions'
-
-            env['LDFLAGS'] += f' -L{arch.ndk_lib_dir}'
-            env['LDFLAGS'] += f' -l{self.stl_lib_name}'
-        return env
-
 
 recipe = KiwiSolverRecipe()

pythonforandroid/recipes/kiwisolver/__init__.py

@@ -6,18 +6,18 @@
 from pythonforandroid.archs import Arch
 from pythonforandroid.logger import shprint
 from pythonforandroid.recipe import Recipe
-from pythonforandroid.util import current_directory, ensure_dir
+from pythonforandroid.util import current_directory
 
 
 class LibLzmaRecipe(Recipe):
 
     version = '5.2.4'
     url = 'https://tukaani.org/xz/xz-{version}.tar.gz'
-    built_libraries = {'liblzma.so': 'install/lib'}
+    built_libraries = {'liblzma.so': 'p4a_install/lib'}
 
     def build_arch(self, arch: Arch) -> None:
         env = self.get_recipe_env(arch)
-        install_dir = join(self.get_build_dir(arch.arch), 'install')
+        install_dir = join(self.get_build_dir(arch.arch), 'p4a_install')
         with current_directory(self.get_build_dir(arch.arch)):
             if not exists('configure'):
                 shprint(sh.Command('./autogen.sh'), _env=env)
@@ -42,7 +42,6 @@ def build_arch(self, arch: Arch) -> None:
                 _env=env
             )
 
-            ensure_dir('install')
             shprint(sh.make, 'install', _env=env)
 
     def get_library_includes(self, arch: Arch) -> str:
@@ -52,7 +51,7 @@ def get_library_includes(self, arch: Arch) -> str:
         variable `CPPFLAGS`.
         """
         return " -I" + join(
-            self.get_build_dir(arch.arch), 'install', 'include',
+            self.get_build_dir(arch.arch), 'p4a_install', 'include',
         )
 
     def get_library_ldflags(self, arch: Arch) -> str: