Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mirrors #493

Open
wants to merge 12 commits into
base: master
Choose a base branch
from
Open

Mirrors #493

wants to merge 12 commits into from

Conversation

fosslinux
Copy link
Owner

@fosslinux fosslinux commented Jan 4, 2025

Here's a first attempt at a mirrors network as per #485.

This is quite basic, and intended to be iterated upon to possibly add things like IPFS/torrents, mirror discovery, etc.

At this point in time, this PR only attempts to implement the following model:

  • mirrors are given by a static list as a configuration variable
  • each mirror has a copy of all distfiles, and creates git archive snapshots

The "sources" format has a change: there is now a Git line supported, which has the format:

git://<Git url> <url to a Git archive snapshot, if one exists, or else an underscore> <checksum of the Git archive snapshot> [filename]

download-distfiles.sh, rootfs.py and helpers.sh have been updated to reflect this.

mirror.sh is an implementation of a mirror creation/maintenance tool, it is intentionally written in POSIX sh (not bash, like we use for most other things), because platform diversity should be encouraged. I would also encourage other mirror implementations to be written eventually.

Alongside that, in this PR I have accordingly swapped out a number of distfiles to get us "closer to upstream";

  • all gnulib snapshots have been changed to git archive snapshots
  • all GitHub snapshots (not release artifacts, which could be non-dynamically generated tarballs) have been changed to git archive snapshots; namely, gnu-autogen-bootstrapping in autogen, guile-psyntax-bootstrapping in guile, kexec-tools
  • other Savannah snapshots, namely Autogen and coreutils, have been changed to git archive snapshots
  • bash-2.05b, bc-1.07.1, bzip2-1.0.8, ed-1.4 have all had their tarballs moved back to upstream, rather than recompressed versions created by various distributions
  • gcc-10.4.0 and gcc-13.1.0 have moved to mirrors.kernel.org for consistency

As of yet unresolved questions;

  • what do we do about CI now? should it use the main mirror network? or should it use a special CI mirror?

mirror.sh Show resolved Hide resolved
@fosslinux fosslinux force-pushed the mirrors branch 2 times, most recently from 675c067 to 10ea6c3 Compare January 13, 2025 08:49
It wasn't ever really useful.
@fosslinux fosslinux force-pushed the mirrors branch 2 times, most recently from 734d494 to 88e9a54 Compare January 13, 2025 09:52
@fosslinux fosslinux force-pushed the mirrors branch 2 times, most recently from 2988476 to f8663d7 Compare January 14, 2025 08:55
@fosslinux fosslinux force-pushed the mirrors branch 2 times, most recently from a7b9e40 to 84a3aad Compare January 18, 2025 22:09
@fosslinux fosslinux force-pushed the mirrors branch 2 times, most recently from f6177ae to e7c1609 Compare January 23, 2025 06:38
@fosslinux fosslinux marked this pull request as ready for review January 23, 2025 22:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants