Merge pull request #2796 from misl6/release-2023.05.21

* Add support for Python 3.10 and make it the default while building hostpython3 and python3 (#2577)

* Add support for Python 3.10 and make it the default while building hostpython3 and python3

* Update bugfix version

* Add PyAV recipe (#2750)

* pyav recipe

* flake8 fix

* Update pythonforandroid/recipes/av/__init__.py

Co-authored-by: Andre Miras <[email protected]>

---------

Co-authored-by: Andre Miras <[email protected]>

* Add debug loggings for identifying a matching dist (#2751)

* Add debug loggings for identifying a matching dist

* Fix lint failure on f-string.

* Adds flag to support home app (launcher) intent-filter on SDL2 bootstrap (#2761)

* Home app functionality

* Cleanup

* Added --home-app to the docs

* Fixed the if statements within the intent-filter

* Use io.IOBase for LogFile (#2763)

This provides reasonable default implementations for common IO class
methods such as isatty().

Closes #2762.

* android_api to integer (#2765)

* android_api to integer

* changes to android_api

* changes to android_api

* fix #2768 smallIconName null can t be compared to String (#2769)

* fix #2768 smallIconName null can t be compared to String

* fix: Remove Importance None of notification which is modification from my own fork

* fix: Readd Importance_NONE in the notification channel as it s also in upstream p4a

* fix tools/liblink: syntax error

regression from 22b7dfe

* Update pyjnius version in recipe (#2791)

* Update Kivy recipe for 2.2.0 (#2793)

* Update Kivy recipe for 2.2.0rc1

* Bump to 2.2.0

* Update CHANGELOG.md and bump version to 2023.05.21

---------

Co-authored-by: Dexer <[email protected]>
Co-authored-by: Andre Miras <[email protected]>
Co-authored-by: BitcoinWukong <[email protected]>
Co-authored-by: Mathias Lindström <[email protected]>
Co-authored-by: Dylan McCall <[email protected]>
Co-authored-by: Benoît HERVIER <[email protected]>
Co-authored-by: SomberNight <[email protected]>

Author: 8 people

CHANGELOG.md

@@ -1,5 +1,45 @@
 # Changelog
 
+## [v2023.05.21](https://github.com/kivy/python-for-android/tree/v2023.05.21)
+
+[Full Changelog](https://github.com/kivy/python-for-android/compare/v2023.02.10...v2023.05.21)
+
+**Closed issues:**
+
+- python [\#2795](https://github.com/kivy/python-for-android/issues/2795)
+- Create APK from PyQt app [\#2794](https://github.com/kivy/python-for-android/issues/2794)
+- psutil/\_psutil\_linux.so" is 64-bit instead of 32-bit [\#2785](https://github.com/kivy/python-for-android/issues/2785)
+- pythonforandroid.toolchain.py: error: unrecognized arguments: --dir [\#2775](https://github.com/kivy/python-for-android/issues/2775)
+- App [\#2774](https://github.com/kivy/python-for-android/issues/2774)
+- org.kivy.android.PythonActivity$NewIntentListener is not visible from class loader java.lang.IllegalArgumentException [\#2770](https://github.com/kivy/python-for-android/issues/2770)
+- Service don t start anymore, as smallIconName extra is now mandatory [\#2768](https://github.com/kivy/python-for-android/issues/2768)
+- Start a background sticky service that auto-restart. [\#2767](https://github.com/kivy/python-for-android/issues/2767)
+- Fail installation [\#2764](https://github.com/kivy/python-for-android/issues/2764)
+- Python exception when using colorlog due to incomplete IO implementation in sys.stderr [\#2762](https://github.com/kivy/python-for-android/issues/2762)
+- AttributeError: 'org.kivy.android.PythonService' object has no attribute 'getComponentName' [\#2760](https://github.com/kivy/python-for-android/issues/2760)
+- https://code.videolan.org not available [\#2758](https://github.com/kivy/python-for-android/issues/2758)
+- Cannot install Python-for-Android [\#2754](https://github.com/kivy/python-for-android/issues/2754)
+- c/\_cffi\_backend.c:407:23: error: expression is not assignable [\#2753](https://github.com/kivy/python-for-android/issues/2753)
+- not install [\#2749](https://github.com/kivy/python-for-android/issues/2749)
+- APK crashes upon launch. logcat error: null pointer dereference \(occurs with imported modules\) [\#2358](https://github.com/kivy/python-for-android/issues/2358)
+- Error occured while building the aplication using buildozer [\#2104](https://github.com/kivy/python-for-android/issues/2104)
+- "Could Not Extract Public Data"  Needs very explicit instructions or feedback to the user [\#260](https://github.com/kivy/python-for-android/issues/260)
+
+**Merged pull requests:**
+
+- Update Kivy recipe for 2.2.0 [\#2793](https://github.com/kivy/python-for-android/pull/2793) ([misl6](https://github.com/misl6))
+- Update `pyjnius` version to `1.5.0` [\#2791](https://github.com/kivy/python-for-android/pull/2791) ([misl6](https://github.com/misl6))
+- fix tools/liblink: syntax error [\#2771](https://github.com/kivy/python-for-android/pull/2771) ([SomberNight](https://github.com/SomberNight))
+- fix \#2768 smallIconName null can t be compared to String [\#2769](https://github.com/kivy/python-for-android/pull/2769) ([brvier](https://github.com/brvier))
+- android\_api to integer [\#2765](https://github.com/kivy/python-for-android/pull/2765) ([kuzeyron](https://github.com/kuzeyron))
+- Use io.IOBase for LogFile [\#2763](https://github.com/kivy/python-for-android/pull/2763) ([dylanmccall](https://github.com/dylanmccall))
+- Home app functionality [\#2761](https://github.com/kivy/python-for-android/pull/2761) ([kuzeyron](https://github.com/kuzeyron))
+- Add debug loggings for identifying a matching dist [\#2751](https://github.com/kivy/python-for-android/pull/2751) ([BitcoinWukong](https://github.com/BitcoinWukong))
+- Add PyAV recipe [\#2750](https://github.com/kivy/python-for-android/pull/2750) ([DexerBR](https://github.com/DexerBR))
+- Merge master into develop [\#2748](https://github.com/kivy/python-for-android/pull/2748) ([misl6](https://github.com/misl6))
+- Add support for Python 3.10 and make it the default while building hostpython3 and python3 [\#2577](https://github.com/kivy/python-for-android/pull/2577) ([misl6](https://github.com/misl6))
+
+
 ## [v2023.02.10](https://github.com/kivy/python-for-android/tree/v2023.02.10) (2023-02-10)
 
 [Full Changelog](https://github.com/kivy/python-for-android/compare/v2023.01.28...v2023.02.10)

doc/source/buildoptions.rst

@@ -71,6 +71,7 @@ options (this list may not be exhaustive):
 - ``--icon``: A path to the png file to use as the application icon.
 - ``--permission``: A permission that needs to be declared into the App ``AndroidManifest.xml``.
   For multiple permissions, add multiple ``--permission`` arguments.
+  ``--home-app`` Gives you the option to set your application as a home app (launcher) on your Android device.
 
   .. Note ::
     ``--permission`` accepts the following syntaxes: 

pythonforandroid/__init__.py

@@ -1 +1 @@
-__version__ = '2023.02.10'
+__version__ = '2023.05.21'

pythonforandroid/bootstraps/common/build/build.py

@@ -498,14 +498,16 @@ def make_package(args):
     with open('project.properties', 'r') as fileh:
         target = fileh.read().strip()
     android_api = target.split('-')[1]
-    try:
-        int(android_api)
-    except (ValueError, TypeError):
+
+    if android_api.isdigit():
+        android_api = int(android_api)
+    else:
         raise ValueError(
             "failed to extract the Android API level from " +
             "build.properties. expected int, got: '" +
             str(android_api) + "'"
         )
+
     with open('local.properties', 'r') as fileh:
         sdk_dir = fileh.read().strip()
     sdk_dir = sdk_dir[8:]
@@ -773,6 +775,8 @@ def create_argument_parser():
         ap.add_argument('--launcher', dest='launcher', action='store_true',
                         help=('Provide this argument to build a multi-app '
                               'launcher, rather than a single app.'))
+        ap.add_argument('--home-app', dest='home_app', action='store_true', default=False,
+                        help=('Turn your application into a home app (launcher)'))
     ap.add_argument('--permission', dest='permissions', action='append', default=[],
                     help='The permissions to give this app.', nargs='+')
     ap.add_argument('--meta-data', dest='meta_data', action='append', default=[],

pythonforandroid/bootstraps/common/build/jni/application/src/start.c

@@ -206,7 +206,7 @@ int main(int argc, char *argv[]) {
   /* inject our bootstrap code to redirect python stdin/stdout
    * replace sys.path with our path
    */
-  PyRun_SimpleString("import sys, posix\n");
+  PyRun_SimpleString("import io, sys, posix\n");
 
   char add_site_packages_dir[256];
 
@@ -224,17 +224,19 @@ int main(int argc, char *argv[]) {
   }
 
   PyRun_SimpleString(
-      "class LogFile(object):\n"
+      "class LogFile(io.IOBase):\n"
       "    def __init__(self):\n"
       "        self.__buffer = ''\n"
+      "    def readable(self):\n"
+      "        return False\n"
+      "    def writable(self):\n"
+      "        return True\n"
       "    def write(self, s):\n"
       "        s = self.__buffer + s\n"
       "        lines = s.split('\\n')\n"
       "        for l in lines[:-1]:\n"
       "            androidembed.log(l.replace('\\x00', ''))\n"
       "        self.__buffer = lines[-1]\n"
-      "    def flush(self):\n"
-      "        return\n"
       "sys.stdout = sys.stderr = LogFile()\n"
       "print('Android path', sys.path)\n"
       "import os\n"

pythonforandroid/bootstraps/common/build/src/main/java/org/kivy/android/PythonService.java

@@ -113,18 +113,20 @@ protected void doStartForeground(Bundle extras) {
 
 	// Unspecified icon uses default.
 	int smallIconId = context.getApplicationInfo().icon;
-	if (!smallIconName.equals("")){
-	    int resId = getResources().getIdentifier(smallIconName, "mipmap",
-						     getPackageName());
-	    if (resId ==0) {
-		resId = getResources().getIdentifier(smallIconName, "drawable",
-						     getPackageName());
-	    }
-	    if (resId !=0) {
-		smallIconId = resId;
-	    }
-	}
-	    
+    if (smallIconName != null) {
+        if (!smallIconName.equals("")){
+            int resId = getResources().getIdentifier(smallIconName, "mipmap",
+                                 getPackageName());
+            if (resId ==0) {
+            resId = getResources().getIdentifier(smallIconName, "drawable",
+                                 getPackageName());
+            }
+            if (resId !=0) {
+            smallIconId = resId;
+            }
+        }
+    }
+
 	if (Build.VERSION.SDK_INT < Build.VERSION_CODES.O) {
 	    // This constructor is deprecated
             notification = new Notification(
@@ -143,9 +145,8 @@ protected void doStartForeground(Bundle extras) {
             // https://stackoverflow.com/questions/47531742/startforeground-fail-after-upgrade-to-android-8-1
             String NOTIFICATION_CHANNEL_ID = "org.kivy.p4a" + getServiceId();
             String channelName = "Background Service" + getServiceId();
-            NotificationChannel chan = new NotificationChannel(NOTIFICATION_CHANNEL_ID, channelName, 
-                NotificationManager.IMPORTANCE_NONE);
-            
+            NotificationChannel chan = new NotificationChannel(NOTIFICATION_CHANNEL_ID, channelName, NotificationManager.IMPORTANCE_NONE);
+
             chan.setLightColor(Color.BLUE);
             chan.setLockscreenVisibility(Notification.VISIBILITY_PRIVATE);
             NotificationManager manager = (NotificationManager) getSystemService(Context.NOTIFICATION_SERVICE);

pythonforandroid/bootstraps/common/build/src/main/java/org/kivy/android/PythonUtil.java

@@ -54,6 +54,7 @@ protected static ArrayList<String> getLibraries(File libsDir) {
         libsList.add("python3.7m");
         libsList.add("python3.8");
         libsList.add("python3.9");
+        libsList.add("python3.10");
         libsList.add("main");
         return libsList;
     }
@@ -73,7 +74,7 @@ public static void loadLibraries(File filesDir, File libsDir) {
                 // load, and it has failed, give a more
                 // general error
                 Log.v(TAG, "Library loading error: " + e.getMessage());
-                if (lib.startsWith("python3.9") && !foundPython) {
+                if (lib.startsWith("python3.10") && !foundPython) {
                     throw new RuntimeException("Could not load any libpythonXXX.so");
                 } else if (lib.startsWith("python")) {
                     continue;

pythonforandroid/bootstraps/sdl2/build/templates/AndroidManifest.tmpl.xml

@@ -76,19 +76,22 @@
                   {% endif %}
                   >
 
-            {% if args.launcher %}
             <intent-filter>
+            {% if args.launcher %}            
                 <action android:name="org.kivy.LAUNCH" />
                 <category android:name="android.intent.category.DEFAULT" />
                 <data android:scheme="{{ url_scheme }}" />
-            </intent-filter>
             {% else  %}
-            <intent-filter>
                 <action android:name="android.intent.action.MAIN" />
                 <category android:name="android.intent.category.LAUNCHER" />
-            </intent-filter>
             {% endif %}
 
+            {% if args.home_app %}
+                <category android:name="android.intent.category.HOME" />
+                <category android:name="android.intent.category.DEFAULT" />
+            {% endif %}
+            </intent-filter>
+
             {%- if args.intent_filters -%}
             {{- args.intent_filters -}}
             {%- endif -%}

pythonforandroid/distribution.py

@@ -2,7 +2,7 @@
 import glob
 import json
 
-from pythonforandroid.logger import (info, info_notify, warning, Err_Style, Err_Fore)
+from pythonforandroid.logger import (debug, info, info_notify, warning, Err_Style, Err_Fore)
 from pythonforandroid.util import current_directory, BuildInterruptingException
 from shutil import rmtree
 
@@ -91,6 +91,7 @@ def get_distribution(
         '''
 
         possible_dists = Distribution.get_distributions(ctx)
+        debug(f"All possible dists: {possible_dists}")
 
         # Will hold dists that would be built in the same folder as an existing dist
         folder_match_dist = None
@@ -100,6 +101,7 @@ def get_distribution(
             possible_dists = [
                 d for d in possible_dists if
                 (d.name == name) and all(arch_name in d.archs for arch_name in archs)]
+            debug(f"Dist matching name and arch: {possible_dists}")
 
             if possible_dists:
                 # There should only be one folder with a given dist name *and* arch.
@@ -115,13 +117,18 @@ def get_distribution(
             if (
                 ndk_api is not None and dist.ndk_api != ndk_api
             ) or dist.ndk_api is None:
+                debug(
+                    f"dist {dist} failed to match ndk_api, target api {ndk_api}, dist api {dist.ndk_api}"
+                )
                 continue
             for recipe in recipes:
                 if recipe not in dist.recipes:
+                    debug(f"dist {dist} missing recipe {recipe}")
                     break
             else:
                 _possible_dists.append(dist)
         possible_dists = _possible_dists
+        debug(f"Dist matching ndk_api and recipe: {possible_dists}")
 
         if possible_dists:
             info('Of the existing distributions, the following meet '
@@ -133,17 +140,24 @@ def get_distribution(
         # If any dist has perfect recipes, arch and NDK API, return it
         for dist in possible_dists:
             if force_build:
+                debug("Skipping dist due to forced build")
                 continue
             if ndk_api is not None and dist.ndk_api != ndk_api:
+                debug("Skipping dist due to ndk_api mismatch")
                 continue
             if not all(arch_name in dist.archs for arch_name in archs):
+                debug("Skipping dist due to arch mismatch")
                 continue
             if (set(dist.recipes) == set(recipes) or
                 (set(recipes).issubset(set(dist.recipes)) and
                  not require_perfect_match)):
                 info_notify('{} has compatible recipes, using this one'
                             .format(dist.name))
                 return dist
+            else:
+                debug(
+                    f"Skipping dist due to recipes mismatch, expected {set(recipes)}, actual {set(dist.recipes)}"
+                )
 
         # If there was a name match but we didn't already choose it,
         # then the existing dist is incompatible with the requested

pythonforandroid/recipes/av/__init__.py

@@ -0,0 +1,25 @@
+from pythonforandroid.toolchain import Recipe
+from pythonforandroid.recipe import CythonRecipe
+
+
+class PyAVRecipe(CythonRecipe):
+
+    name = "av"
+    version = "10.0.0"
+    url = "https://github.com/PyAV-Org/PyAV/archive/v{version}.zip"
+
+    depends = ["python3", "cython", "ffmpeg", "av_codecs"]
+    opt_depends = ["openssl"]
+
+    def get_recipe_env(self, arch, with_flags_in_cc=True):
+        env = super().get_recipe_env(arch)
+
+        build_dir = Recipe.get_recipe("ffmpeg", self.ctx).get_build_dir(
+            arch.arch
+        )
+        self.setup_extra_args = ["--ffmpeg-dir={}".format(build_dir)]
+
+        return env
+
+
+recipe = PyAVRecipe()

pythonforandroid/recipes/av_codecs/__init__.py

@@ -0,0 +1,11 @@
+from pythonforandroid.toolchain import Recipe
+
+
+class PyAVCodecsRecipe(Recipe):
+    depends = ["libx264", "libshine", "libvpx"]
+
+    def build_arch(self, arch):
+        pass
+
+
+recipe = PyAVCodecsRecipe()

pythonforandroid/recipes/hostpython3/__init__.py

@@ -35,7 +35,7 @@ class HostPython3Recipe(Recipe):
         :class:`~pythonforandroid.python.HostPythonRecipe`
     '''
 
-    version = '3.9.9'
+    version = '3.10.10'
     name = 'hostpython3'
 
     build_subdir = 'native-build'

pythonforandroid/recipes/kivy/__init__.py

@@ -22,12 +22,12 @@ def is_kivy_affected_by_deadlock_issue(recipe=None, arch=None):
 
 
 class KivyRecipe(CythonRecipe):
-    version = '2.1.0'
+    version = '2.2.0'
     url = 'https://github.com/kivy/kivy/archive/{version}.zip'
     name = 'kivy'
 
     depends = ['sdl2', 'pyjnius', 'setuptools']
-    python_depends = ['certifi']
+    python_depends = ['certifi', 'chardet', 'idna', 'requests', 'urllib3']
 
     # sdl-gl-swapwindow-nogil.patch is needed to avoid a deadlock.
     # See: https://github.com/kivy/kivy/pull/8025

pythonforandroid/recipes/pyjnius/__init__.py

@@ -6,7 +6,7 @@
 
 
 class PyjniusRecipe(CythonRecipe):
-    version = '1.4.2'
+    version = '1.5.0'
     url = 'https://github.com/kivy/pyjnius/archive/{version}.zip'
     name = 'pyjnius'
     depends = [('genericndkbuild', 'sdl2'), 'six']

pythonforandroid/recipes/python3/__init__.py

@@ -56,7 +56,7 @@ class Python3Recipe(TargetPythonRecipe):
         :class:`~pythonforandroid.python.GuestPythonRecipe`
     '''
 
-    version = '3.9.9'
+    version = '3.10.10'
     url = 'https://www.python.org/ftp/python/{version}/Python-{version}.tgz'
     name = 'python3'
 
@@ -70,14 +70,16 @@ class Python3Recipe(TargetPythonRecipe):
 
         # Python 3.8.1 & 3.9.X
         ('patches/py3.8.1.patch', version_starts_with("3.8")),
-        ('patches/py3.8.1.patch', version_starts_with("3.9"))
+        ('patches/py3.8.1.patch', version_starts_with("3.9")),
+        ('patches/py3.8.1.patch', version_starts_with("3.10"))
     ]
 
     if shutil.which('lld') is not None:
         patches = patches + [
             ("patches/py3.7.1_fix_cortex_a8.patch", version_starts_with("3.7")),
             ("patches/py3.8.1_fix_cortex_a8.patch", version_starts_with("3.8")),
-            ("patches/py3.8.1_fix_cortex_a8.patch", version_starts_with("3.9"))
+            ("patches/py3.8.1_fix_cortex_a8.patch", version_starts_with("3.9")),
+            ("patches/py3.8.1_fix_cortex_a8.patch", version_starts_with("3.10"))
         ]
 
     depends = ['hostpython3', 'sqlite3', 'openssl', 'libffi']
@@ -96,6 +98,7 @@ class Python3Recipe(TargetPythonRecipe):
         'ac_cv_file__dev_ptc=no',
         '--without-ensurepip',
         'ac_cv_little_endian_double=yes',
+        'ac_cv_header_sys_eventfd_h=no',
         '--prefix={prefix}',
         '--exec-prefix={exec_prefix}',
         '--enable-loadable-sqlite-extensions')

pythonforandroid/tools/liblink

@@ -35,6 +35,7 @@ while i < len(sys.argv):
 
     if opt.startswith(
             ("-I", "-isystem", "-m", "-f", "-O", "-g", "-D", "-R")):
+        continue
 
     if opt.startswith("-"):
         print(sys.argv)