GHDL

Download and keep self-contained executables from Github Releases updated.

Install

pip install ghdl

Additionally, it requires your system to have sqlite, libmagic and tools relevant to archives like tar, gzip, unzip etc. installed.

About

More and more projects are doing self-contained multi-platform/architecture binary releases at github. While distro provided packages remain the best idea, they are not a portable/zero-maintenance solution. If you hop between machines running on different hardware/OS/architecture, it’s convenient to just download binaries directly.

But doing so and keeping them updated manually is tedious. With this tool you can automate the task by encapsulating the logic in a config file which you can then carry aroud, maybe even hook up to a cron and forget.

Config

The config file should be provided at ~/.config/ghdl/config. It looks like:

(repo "containers/crun"
  :asset-filter "static")

When you run ghdl, it will download the asset with “static” in its name, extract the archive if needed, find the binary, and place it in ~/.local/bin/ (by default). And next time you run ghdl it won’t be downloaded again, unless upstream releases new version in-between.

Options

These are the options you can specify:

:asset-filter (required)

A regular expression that can uniquely identify an asset from a list of them in the latest release section of the target repo. Matching is done case-insensitively.

:name

Name the binary will be saved as. Default is to infer it from repo name, so when the repo is “containers/crun” the name will by default be “crun”.

:basename-glob

When asset is an archive, ghdl automatically extracts it. But it needs to know which file to install, as there are usually multiple files in the archive. By default it can filter out only binaries using libmagic, and by default it looks for a binary that matches ”*{name}*” glob pattern which almost always works (name is what’s described above). So in practice, you will likely need to set this very rarely. For example, below you could omit it because the binary is named “devd” which is matched by the glob “∗devd∗” anyway:

(repo "cortesi/devd"
  :basename-glob "devd" ;; Not necessary
  :asset-filter "linux64")

And in the following case, again the binary name “kompose-linux-amd64” is already covered by glob “∗kompose∗”:

(repo "kubernetes/kompose"
  :basename-glob "kompose*" ;; Not necessary
  :asset-filter "linux-amd64.*gz$")

Sometimes the default is too broad. For example, in case of repo ”github/hub” the binary is named just “hub” which isn’t a problem, but the glob “∗hub∗” matches not just the binary but some other doc file too. Except that’s where libmagic comes in and filters out all non-application mimetypes.

(repo "github/hub"
  :basename-glob "hub" ;; Again not necessary
  :asset-filter "linux-amd64")

So in practice, it’s probably going to be exceedingly rare when this setting will be called for action.

:strip

Whether to run strip on the downloaded binary or not, default is True. But you may need to set it to False sometimes, e.g. say for appimages:

(repo "neovim/neovim"
  :url-filter "appimage$"
  :strip False)

Stripping can reduce the size of binaries considerably (here the /bin/strip is invoked without argument), but this might not be without trade-off. I have once seen a distro turn off stripping for Golang binaries citing strange behaviour. I have never encountered any problem myself though, so I am keeping this True by default.

:prerelease

You can also choose to live in the edge, some packages for some reason choose to release as pre-release only anyway (so far).

(repo "borkdude/babashka"
  :name "bb"
  :asset-filter "static"
  :prerelease True)

:pin

Sometimes you might want to pin a particular repo to a particular architecture as that’s all they support in github releases. Here pinning can be useful to create portable configuration, as ghdl simply won’t do anything when run on anything different. For example,

(repo "LukeChannings/deno-arm64"
  :name "deno"
  :pin "aarch64"
  :asset-filter "deno")

As this is an aarch64 specific project, pinning ensures the recipe won’t even be run on say x86_64, so you are spared of the failure message.

:release-filter

Some projects release multiple unrelated kind of artefacts under same repo. While we still can’t have multiple entries for same repo in ghdl, if you want to filter a particular kind of release amidst many, this is helpful. For example biomejs/biome repo releases VSCode Extension and CLI under same repo, if we just want the CLI, we need to first find the matching release:

(repo "biomejs/biome"
  :release-filter "\\bCLI\\b" ;; regex with word boundary
  :asset-filter f"linux-x64") ;; this will now look into the release found by :release-filter

Advanced Configuration

The goal is to eventually define something that hopefully can continue to work cross-platform. So ghdl config file is actually Hy code, which means one can do whatever they can in python, and some useful python modules (re, platform etc.) are loaded already:

(setv my-os (platform.system)
      my-arch (platform.machine))

(setv arch-pattern
  (cond (= my-arch "x86_64") "(?:x86[-_]64|x64|amd64|64bit)"
        (= my-arch "aarch64") "(?:arm|aarch)64"))

;; Many golang projects follow this pattern
(setv os-arch f"{my-os}[-_.]{arch-pattern}")

;; Many rust projects follow this pattern
(setv rust f"{my-arch}.*?{my-os}")

;; Finally define the repos
(repo "caddyserver/caddy"
  :asset-filter f"{os-arch}.*gz")

(repo "sharkdp/bat"
  :asset-filter rust)

If that seems ugly or inadequate, :asset-filter can also be a function (of type String -> Bool):

(repo "smallhadroncollider/taskell"
  :asset-filter
    (fn [asset]
      (and
        (in (str.replace (platform.machine) "_" "-") asset)
        (in (str.lower (platform.system)) asset)
        (str.endswith asset ".gz"))))

Additional Config

You really should user your own API token if possible. Unauthorized API is only limited to 60 calls per hour, so if you have more than 60 repos you need to use a token anyway.

(config
  :location "~/.local/bin/"
  :token "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx")

Sample config

The repo contains a sample config file as an example where I am tracking 140+ projects (as of writing this). Most of them has been well behaved and consistent since creating ghdl.

TODO/Limitation/Ideas

• Some programs use same repo to release different parts of project (e.g. dnote does it for server and cli), ghdl can choose one of them, but not multiple at the same time.
• Some projects include multiple binaries. That’s not yet supported, though is probably simple to add.
• Windows/MacOS doesn’t work yet (so much for cross-platform), although fix should be simple; I just don’t know various xdg path equivalents in it nor do I have motivation or means to test. However one needs to have tar, gzip etc. in the path which means the likes of msys2, git bash, WSL or whatever, and in those it just might work ootb.
• Downloads from github could be slow in some parts of the world, nothing could done about that apart from maybe switching to an external downloader (like aria2) from current pure python one.