Set up automated packaging for various distros

[jeaye]

Recently, I set up automatic releases, based on the last successful commit: https://github.com/jank-lang/jank/releases

These are just a tarball of the result of using cmake to install all necessary files. In order to make this easy to actually install on another machine, we'd really benefit from continuous packages for various distros and OSs. I'd say start first with whatever you use, if you want to do this.

• arch
• debian
• nix
• macOS (homebrew)
• freebsd
• windows (chocolatey)

APE

I've been chatting with the Cosmo/APE folks about what would be required to get jank compiling in a portable way. Looks like, for now, we have:

• Switch to libc++
• Move away from vcpkg (use submodules)
• Statically compile all deps (include libc)
• Compile using the Cosmo toolchain and test basic functionality
• Look for JIT issues; they're expected

[lafka]

As you mentioned in #35 cling/clang/llvm seems to be tightly coupled. From the description in https://root.cern/cling/cling_build_instructions/ it seems to be possible to use a unpatched version of llvm which would simplify the setup.

How closely tied is Jank to to a specific version of cling? Can cling be removed all together, or can it use any precompiled binary[1][2]? If cling can use llvm provided by the distro it would simplify the build process.

Have you considered using AppImage, Snap or similar instead of distro packaging?

[1] https://root.cern.ch/download/cling/
[2] https://launchpad.net/~ppa-verse/+archive/ubuntu/xeus-cling - someone has packaged cling for ubuntu atleast

[jeaye]

As you mentioned in #35 cling/clang/llvm seems to be tightly coupled. From the description in https://root.cern/cling/cling_build_instructions/ it seems to be possible to use a unpatched version of llvm which would simplify the setup.

How closely tied is Jank to to a specific version of cling? Can cling be removed all together, or can it use any precompiled binary[1][2]? If cling can use llvm provided by the distro it would simplify the build process.

Have you considered using AppImage, Snap or similar instead of distro packaging?

[1] https://root.cern.ch/download/cling/ [2] https://launchpad.net/~ppa-verse/+archive/ubuntu/xeus-cling - someone has packaged cling for ubuntu atleast

Cling has some precompiled binaries, and some distros package Cling as well, but I've found that a lot of the pre-packaged Cling distributions are actually lacking the shared/static libs needed to use Cling as a library. Instead, they focus on using Cling as a CLI REPL. An example of this would be NixOS, but I believe the AUR package has the same issue.

Ultimately, as I mentioned in my latest blog post, I'm switching to clang-repl. The creator of Cling, Vassil, is trying to move people who aren't tied to ROOT onto clang-repl, since it's where the majority of future development is going to happen. I've been working with him as part of the migration and he's been helping to resolve some of the issues I've identified.

If you're at all familiar with Cling/Clang/LLVM, and can help work through the issues jank has with them, definitely hit me up. That's an area I haven't been able to put much time into while I'm focusing on achieving parity with Clojure's compiler, runtime, and stdlib.

Have you considered using AppImage, Snap or similar instead of distro packaging

Yeah, I'm chewing on this. Nothing is decided for how jank will be distributed yet, but my preference would be for it to be as native as possible in each distro (i.e. not bundling its own libc, llvm, etc). I've never heard a good thing about snap, from consumers, but the appimages I've used have been easy enough.

Rust has the rustup tool, which is part of the distro. It can be used to then pull down various versions of Rust, switch to a different version, etc. It may make sense to go down this route sooner rather than later. If we did, jankup could be in each distro, as a script, which would then pull down appimages or something like that.

Ultimately, I'd like to discuss more with people who have experience shipping native executables to various distro package managers. All of my open source tools, so far, have been distributed in other ways. If you have experience with this sort of thing, please do stick around and share your insights.

[lafka]

If you're at all familiar with Cling/Clang/LLVM, and can help work through the issues jank has with them, definitely hit me up. That's an area I haven't been able to put much time into while I'm focusing on achieving parity with Clojure's compiler, runtime, and stdlib.

Unfortunately i have never heard of cling before today so not much help there.

Have you considered using AppImage, Snap or similar instead of distro packaging

Yeah, I'm chewing on this. Nothing is decided for how jank will be distributed yet, but my preference would be for it to be as native as possible in each distro (i.e. not bundling its own libc, llvm, etc). I've never heard a good thing about snap, from consumers, but the appimages I've used have been easy enough.

Ultimately, I'd like to discuss more with people who have experience shipping native executables to various distro package managers. All of my open source tools, so far, have been distributed in other ways. If you have experience with this sort of thing, please do stick around and share your insights.

I don't have positive to say about the snap either, the effort is better spent working on switching to clang-repl, after that it should be straight forward to package for distros. Looking at packages in CMakeLists there some (atleast folly, BDWgc) which are not packaged in debian so that may require additional steps. If you give a heads up here once clang-rpl is landed i'll definitely go through and rebuild all from scratch again. Atleast packaging for alpine and arch is straight forward.

[jeaye]

I don't have positive to say about the snap either, the effort is better spent working on switching to clang-repl, after that it should be straight forward to package for distros. Looking at packages in CMakeLists there some (atleast folly, BDWgc) which are not packaged in debian so that may require additional steps.

Those are only build-time dependencies for jank, so we shouldn't need to worry about them. They're brought in by vcpkg, built from source, and statically linked, when building jank. Though it's not perfect, we can rely on vcpkg to ensure they're portable enough to build wherever jank is needed.

If you give a heads up here once clang-rpl is landed i'll definitely go through and rebuild all from scratch again. Atleast packaging for alpine and arch is straight forward.

Will do! The port to clang-repl is going to simplify so much. :)

[jeaye]

Following up here, since jank has moved to Clang/LLVM 19 (note: 19 is currently unreleased). All build documentation has been updated and local install looks to be working.

If you're still interested in getting us going with some distro packaging, that would be amazing. ❤️

[lafka]

Is this the correct documentation https://github.com/jank-lang/jank/blob/main/compiler+runtime/doc/build.md? I'm guessing clang-repl is not available yet and we're required to build own copy of clang

[jeaye]

Is this the correct documentation https://github.com/jank-lang/jank/blob/main/compiler+runtime/doc/build.md? I'm guessing clang-repl is not available yet and we're required to build own copy of clang

It's correct and up to date!