Android Backend

[AmSmart]

I was trying to get this up and running with MAUI on Android but as far as I can tell, only desktop support is available for the time being. Will the maintainers be open to a PR that merges android CPU support in?

[martindevans]

Android support would be great!

If you're interested in working on it I'd suggest starting out by modifying compile.yml to produce Android binaries. That script is what we use to produce all of the binaries we distribute. Once that's merged we can start looking at what modifications are required for runtime support (if any).

[AmSmart]

Sounds good, I'll start working on this

[DGdev91]

So.... how' the progress on this?

Some time ago i managed to build up a working example on Android (just an experiment to see if it was possible to run an LLM inside Godot Engine, using the c# backend), just by building myself the libraries and loading them using the native lib classes.

The llama.cpp version used for v.0.15 seems like to have a bug wich makes it impossible to compile on android.
The current upstream builds normally, but i can't make it work, not sure if i'm doing something wrong or it's just because the newer versions introduced some breaking changes (i know, i'm supposed to only use that exact version)

[martindevans]

I introduced the changes into the build pipeline before 0.15 (see here) but they're currently disabled, because when I came to do the 0.15 build it failed on Android. I think it was a temporary upstream issue with Android builds, so I'm expecting to be able to enable that for 0.16.

What we don't have yet is any runtime loading of those binaries. There needs to be an item which dynamically detects when we're on Android, checks what CPU it is running on, and loads the right binaries (see here). That's the next piece of work to be done, if you're interested in contributing towards Android support :)

[DGdev91]

I introduced the changes into the build pipeline before 0.15 (see here) but they're currently disabled, because when I came to do the 0.15 build it failed on Android. I think it was a temporary upstream issue with Android builds, so I'm expecting to be able to enable that for 0.16.

What we don't have yet is any runtime loading of those binaries. There needs to be an item which dynamically detects when we're on Android, checks what CPU it is running on, and loads the right binaries (see here). That's the next piece of work to be done, if you're interested in contributing towards Android support :)

Yes, most likely is the same issue i encountered while attempting to compile the android libs for 0.15. i'm able to compile normally now, so i guess it got fixed upstream.

Anyway... before i'm able to help, or at least give that a try, i need a working example. and right now, i'm having some troubles.
On the old version 0.10.0 i didn't had to do much else, i just copied the libs on the output folder for the c# code and those got loaded. i didn't even had to use "NativeLibraryConfig.Instance.WithLibrary("");" (wich now should be different for llama and llava, but whatever....)
I guess that's because of the version not exactly the same, but can easly be something else