Support for integrating external aar files

[Jasper-Bekkers]

Alright so I'm not sure if this is the right repository to post this on but I figured it's as good as any. This ties in a few different things from a few different repositories, all with a sort of common goal: I'd like to use xbuild without Gradle to build applications that contain both Rust and Kotlin (or Java, but in my case I'm focusing on Kotlin) code.

🎒 Background

This may or may not be madness, but at Traverse we're mostly a native Rust shop and we're very comfortable with the Rust, it's ecosystem and build tools, so introducing additional tools or installable requirements on top of "just rust" is always a big issue in terms of workflow. This may or may not be a smart idea tho, as we'll get back to later.

To develop any kind of Android application requires integration with the system libraries and its broader Java ecosystem. In my case specifically this problem arose when I tried getting our NativeActivity application to spawn a wry WebView in the app.

I think @dvc94ch has tried getting this up and running before also (tauri-apps/wry#21) but I'm not entirely sure where this ended up (though I'd be interested since effectively this is still what I'm trying to achieve). However, this issue is a bit broader.

🎮 GameActivity

And so, instead of running our application as a NativeActivity I went on to switch it to being a GameActivity (which inherits from AppCompatActivity - which to my understanding should be able to spawn a wry WebView). However, I also decided to do this in hard mode: I don't know Gradle, and I don't know much about the android ecosystem in general so I dove in head-first. I converted our app and just attacked the problems head on as I found them.

🔬maven-rs

I downloaded packages from Maven repositories, extracted them, stitched them together etc. I built this on top of the wonderful maven-toolbox-rs and have republished the changes as maven-rs since I consider this crate to be quite a deviation from the maven-toolbox. It can download packages both from the google maven repository (where the AppCompatActivity is hosted) and the Maven Central repository (where some of the Kotlin dependencies are hosted).

So far so good; we download jar and aar files, extract them somewhere and then pas them along to kotlin-rs

🐨 kotlin-rs

Alright so having gathered all this up, the next logical step would be to pass those dependencies on to the Kotlin compiler, so I created a small crate kotlin-rs that's similar in spirit to cc-rs but for Kotlin code: https://github.com/Traverse-Research/kotlin-rs it receives the packages in it's class path and all is fine and everybody is happy.... right?

😅 The current problems

So right now this mostly kinda works, as long as it's all super simple. If my kotlin code inherits from Activity and links in only android.jar everything is fine. So where does it go wrong? And why and I posting this on this repository instead of somewhere in else?

Resources, specifically, in order to link everything together, aar files can contain resources (R.txt file defines them, res/ folder then fills them in). During a regular aapt2 compile step of the aar file you'd create an intermediate file which would have a resource table associated with it, and in turn this can then be linked together with aapt2 link into a big resource table - ultimately resources.arsc.

xbuild, when building a regular apk file, internally will also generate this resource table (and it's spending quite a bit of lines of code on doing so). However, one thing that it doesn't (seem to) do, or a least that I don't understand it to be doing, is merging several resource tables together like how aapt2 link would, and this is where I'm stuck.

Currently, going down this path, it's impossible to actually build an apk with AppCompatActivity without Gradle, because even though we can get all the maven packages, link them, generate their class paths and whatever else is needed, we can't get the .apk to start: it's giving NullReferenceExceptions to String.length because it's missing all of the resource that the various aar file have defined in them.

🏃‍♂️ Next steps

For me this is mostly where this project will end for now. I decided to write this all down for a few reasons: to document the process and progress, and to see if other folks find this trajectory interesting as well and could potentially pick it up where I've left off. There's plenty of work remaining but for now the list is relatively short;

1. Create the code to merge together resource files from different aars
2. Fix maven-rs to not just extract the jar file, but also the resources so they can be merged.

[dvc94ch]

I built a maven package manager too, it's the mvn folder, but dependency resolution proved tricky since I don't think maven uses semver. I kept getting incompatible dependencies and then at some point gave up. I'm no expert in java/maven either so...

Being able to build rust/kotlin/java without gradle and linking resource files would be a major step forward in the android/rust ecosystem. I think it's even possible to compile wasm to jvm which would allow even better integration of rust into the android ecosystem.

But as you already noticed it's a major project requiring substantial time investment to build and maintain.

Also building aab's and signing them would also need more work for a complete rust flow.

[Jasper-Bekkers]

I'll check it out - thanks! Quite unfortunate that I missed that the first time around since it seems to be doing more or less the same thing.

I think @MarijnS95 also had some ideas around how to get external crates to contribute to the dexing step that needs to happen as a "post build" step by setting things up in a similar to how cargo::rerun-if-changed works currently.

I do indeed think these are major projects, so maybe turning this into more of a roadmap style issue would help? We could see if it would either attract community contributors or it would serve as a reminder / notepad for ourselves so that when some time comes up I can go back and look what still needs to happen.