Please consider maintaining the Emacs-Lisp files in a separate repository #5101

tarsius · 2022-11-09T18:33:58Z

tarsius
Nov 9, 2022

Please consider maintaining the Emacs-Lisp files in a separate repository

This repository contains one or more Emacs-Lisp libraries, which are distributed on Melpa for easy installation using Emacs' official package manager (package.el) and are also mirrored on the Emacsmirror.

This project/repository is about much more than just the elisp libraries. These libraries are tucked away in some "contrib" or "utilities" directory. The maintainers of the project as a whole likely care little about these libraries, which makes it a less than optimal place to maintain them. The maintainers will always have something more important to deal with and potential contributors are likely scared away by the additional processes and conventions that such a large project needs, but which are overkill when looking at the elisp libraries in isolation.

I would like to propose that the Emacs-Lips libraries are maintained in a separate repository. Doing that would solve a few issues and have additional benefits:

Distributing a package that originates in such a large repository puts additional strain on the Melpa server. Fetching the repository takes longer and they require more space. We are already trying to deal with the latter, but not very successfully and this leads to special-cases in the tooling, which come with additional maintenance costs. See Strategy for huge repositories melpa/melpa#5361, also for some past discussion.
I also maintain the Emacsmirror and here the costs are much higher than for Melpa. The Emacsmirror does not just distribute installable snapshots of a package, as Melpa does.

Instead it mirrors the upstream repository. But in cases like this, where the repository contains much more than just the elisp libraries, it filters the history, using tools such as git filter-repo and git subtree. Unfortunately these tools are not very reliable and highly inefficient when continuously rewriting large repositories like this one.

Currently this affects just 33 out of 8485 on the Emacsmirror, and the disproportionate amount of work that it takes to maintain the tooling for these special-cases is just not worth it. So I have decided to phase out support, but not before making an effort to get as many of these Emacs package slit out of the repositories they are currently being maintained in.
For maintainers of this repository any issue about the elisp libraries is likely just more noise. They have better things to do.
On the other hand, the original authors of the elisp libraries and people who have contributed to them, likely have notifications for this repository disabled because they only contribute to the elisp parts and there would just be too much noise otherwise. So issues and pull-requests by users are likely to go unnoticed for a long time.

The author and contributors likely have to be explicitly summoned, as I am doing now: @dottedmag @yasuyk
Potential contributors are discouraged when their pull-requests are being "ignored" or they might be discouraged by the additional requirements imposed by the project as a whole, and not even try to contribute as a result.
Users who install elisp packages by manually cloning their repositories and then manually adding the appropriate directories to the load-path, are forced to clone a huge repository just to get their hands on a few elisp libraries in some subdirectory. (Or they can use the filtered repositories on the Emacsmirror, but as I mentioned, those are going away.)

So in summary, I think there are many downsides to maintaining the elisp libraries in this repository. I am not sure there are any upsides, except that it is the status quo.

Moving the elip libraries out of here is some work, that someone has to do, but most of that work is already done. You can just clone the respective repository from the Emacsmirror (at https://github.com/emacsmirror/cfengine-code-style) and publish it somewhere else. (Please check that no recent history is missing and if the mirror repository is not uptodate, then ping me.)

Once you have done that tell me about it here, and I will take care of updating Melpa and the Emacsmirror.

Thanks for considering!
Jonas