General question about using binaries / building neovide with ebuild

[tfemby]

Hi there, I have been using your skia-safe package and by extension skia-bindings as a component of building neovide. During my experimentation with trying to use skia binaries, I came across an issue where the bindings config script was requesting a definitions file despite gn and ninja never having been called.

On your README, it says that using the binaries causes a skip of the skia build and bindings generation altogether. This doesn't seem to be the actual behavior of the build script.

The build.rs has the following logic that it seems to bump into when I have a export SKIA_LIBRARY_SEARCH_PATH=/path/to/binaries.

if let Some(search_path) = env::skia_lib_search_path() {
    println!("STARTING BIND AGAINST SYSTEM SKIA");

    binaries_config.import(&search_path, false).unwrap();

    let definitions = skia_bindgen::definitions::from_env();
    generate_bindings(
        &features,
        definitions,
        &binaries_config,
        &source_dir,
        cargo::target(),
        None,
    );
} else {

I'm not super sure if I'm not reading something correctly in the README and this is completely on me. I've been able to successfully build skia and generate bindings with your build script which I greatly appreciate.

If this is the intended behavior, would it be possible to include the definitions file into the binary releases? This would cut down my build times by a good 4:30ish min.

[pragmatrix]

Hmm, may be there is a misunderstanding, or I do not understand the problem because I lack detailed knowledge of the neovide packaging / build process, but if the crate is used, matching binaries are downloaded by skia-binding's build.rs from the skia-binaries repository, e.g. from: https://github.com/rust-skia/skia-binaries/releases/tag/0.64.0 if they are available for the target platform.

for which the statement:

On your README, it says that using the binaries causes a skip of the skia build and bindings generation altogether.

is true.

The build against SKIA_LIBRARY_SEARCH_PATH is not recommended, because Skia's API is in constant flux and building libraries that match the bindings is hard. Besides of creating a matching configuration and extracting the resulting definitions, we add a number of additional changes, e.g.: google/skia@chrome/m115...rust-skia:m115-0.63.2

If you need build.rs to pick up matching binaries from somewhere else, because of offline builds or an unreliable Internet connection, consider using SKIA_BINARIES_URL env variable with the file:// prefix, see #645 and #685 (this isn't documented as it seems).

If you'd like to use a repository clone, you can enforce the download of the binaries by setting FORCE_SKIA_BINARIES_DOWNLOAD.

And this might be related: #479

[tfemby]

Awesome! Thanks for the answer and lending me a your time :)

I wrote an ebuild since I'm on Gentoo Linux and the build environment is sandboxed (no internet access) outside the initial download phase. Currently I'm grabbing 0.62.0 and pulling in the corresponding skia and depot_tools that are listed in the Cargo.toml from skia-bindings. Unfortunately, the skia-bindings build script can't run it's git/https download because of the sandbox environment.

Gentoo creates vendor tarballs and handles all of that within the build sandbox.

So my build process involved me changing the Neovide Cargo.toml to require skia-safe="0.62.0" instead of skia-safe="0.52.0". I'm handling the the depot_tools along with the skia mirror you're hosting by pulling those in in the download phase then moving them over during the unpacking phase.

Here's an example of what I'm doing to get this to work.

https://codeload.github.com/rust-skia/skia/tar.gz/m113-0.61.8 -> skia.tar.gz
https://codeload.github.com/rust-skia/depot_tools/tar.gz/73a2624 -> depot_tools.tar.gz

...
my_skia="skia-m113-0.61.8"
my_depot="depot_tools-73a2624"

mv "${WORKDIR}/${my_skia}" "${CARGO_HOME}/gentoo/skia-bindings/skia"
mv "${WORKDIR}/${my_depot}" "${CARGO_HOME}/gentoo/skia-bindings/depot_tools"

I'm also setting up the build environment using the these environment variables.

export CLANGCC=clang
export CLANGCXX=clang++
export SKIA_SOURCE_DIR="${my_lskia}"
export SKIA_USE_SYSTEM_LIBRARIES=true
export SKIA_NINJA_COMMAND=/usr/bin/ninja
export SKIA_GN_COMMAND=/usr/bin/gn
export BINDGEN_EXTRA_CLANG_ARGS=--target=x86_64-pc-linux-gnu

Each one of these env's required me trawling through this projects issues page and going through how the build skia-bindings build system works; which btw, I really appreciate how nice and tidy it is to 0.52.0, it's way easier to follow along.

Regarding the skia fork, I somewhat figured that you're running your own because it's a Google product linked to Chromium. Googles own git is as busy as when I look at the Linux git page. I'm however honestly surprised just upgrading to a newer tag didn't result in any weird bugs (yet) because of this.

Thanks for linking #479 as well as #645 and #685 . I've admittingly read through many of the issues filed as I came across so many different problems building Skia. I actually got the SKIA_USE_SYSTEM_LIBRARIES idea from reading through #479 a couple of days ago; admittingly got some linking errors which I alleviated via patching the Neovide build.rs.

I do feel that #645 as well as #685 may come in handy for my particular set of needs. I'm going to take a break for a couple of days from working on this since I've got something to work. I'll let you know how adding in the SKIA_BINARIES_URL env var goes. These two in particular would be great adds to the README for skia-bindings and would likely save others who might be working in an offline environment.

[tfemby]

Sorry for the late response. I took a look through #645 and #685 properly. Building based on the binaries worked flawlessly :)

So to explain what I did for this to work.

I fetched the corresponding binary to the skia-bindings version (0.62.0 in this case). Neovide calls for features = ["egl", "gl", "textlayout", "wayland", "x11"] which made determining the correct binary rather easy. On the binary's page there is
skia-binaries-8cf4841aefdb295709e9-x86_64-unknown-linux-gnu-egl-gl-svg-textlayout-wayland-x11.tar.gz which has all of the above + svg . This is close enough to what neovide is asking for.

So to the ebuild. I added the following into my SRC_URI (the download phase of the build script).

SRC_URI+="https://github.com/rust-skia/skia-binaries/releases/download/0.62.0/skia-binaries-8cf4841aefdb295709e9-x86_64-unknown-linux-gnu-egl-gl-svg-textlayout-wayland-x11.tar.gz -> skia-binaries-0.62.0.tar.gz"

Then later on during the compile phase of the script, I had to set the following:

export SKIA_BINARIES_URL="file://${DISTDIR}/skia-binaries-0.62.0.tar.gz"

One interesting quirk which I found was that the ebuild unpacks skia-binaries within $WORKDIR. Pointing $SKIA_BINARIES_URL to the unpacked directory caused an error. The workaround was just to point to the tar.gz file.

I added a few link args into the neovide build.rs because cc wasn't able to find them. Pretty simple to do with a simple patch.

    println!("cargo:rustc-link-arg=-lexpat");
    println!("cargo:rustc-link-arg=-lz");
    println!("cargo:rustc-link-arg=-lpng16");
    println!("cargo:rustc-link-arg=-ljpeg");
    println!("cargo:rustc-link-arg=-lwebp");
    println!("cargo:rustc-link-arg=-lGL");
    println!("cargo:rustc-link-arg=-lEGL");

The only feedback that I would give is that I wish that there could be a function that skips the unpacking part of the download. Could be an env var such as SKIA_UNPACKED_BINARIES or something similar. For now though, I'd say that this is all worked out.

Thank you Pragmatrix for giving me your time with such a helpful response :)

[tfemby]

Oh and one more thing. I said earlier that building skia + generating the bindings slowed down build times by a lot. Using the binaries, this is what I get. Prior to this was average of 5m25s - 5m35s on average. I miscalculated the speedup (I said 4m 30s faster using binaries earlier) but 3m is still excellent!

[pragmatrix]

Thank you a lot for reporting back you results. I am glad that it worked out. This is a valuable contribution, and I hope that other rust-skia users can learn from that.