We have to federate git hosting services, asap

[fakefred]

Outraged by GitHub's or the US's or whoever's decision, I proposed a federated approach to git hosting and interactions between developers, which consists of a network of decentralized nodes, possibly as a middleware or plugin to existing git self-hosting software, enabling a neutral, diverse and non-discriminative git community, while retaining the choice for git instances to remain isolated. >> https://blog.fkfd.me/4​

[fakefred]

@1995parham I feel sorry for your being marginalized by the "agile" SaaS provider under the influence of US government. We need liberty, not a platform that rules us all with arbitrary but inhumane policies. That's why I'm advocating federated git hosting. Therefore, instead of "support" (though I do support this movement), I would rather suggest a "proposal" label for this issue. Various ideas can potentially form the new roadmap of open-source software development, where free is literally free.

Best Regards,
@fakefred

[1995parham]

@fakefred You are right, I have changed the label to the proposal. sorry for the miss understanding. I will be glad to hear more about your idea.

[fakefred]

@1995parham Thank you for the favor. I will try my best to spread awareness.

[ShalokShalom]

This already exists and its developers surely are happy to get help 🤗

All repos in git ssb are quasi private by default and the current way is that you accept contacts optionally, so they can see the repos you share.

In order to gain some public available service like Github, some of us will host lots of repos and accept all/lots of incoming contacts, I imagine.

[fakefred]

@ShalokShalom Glad to learn about git-ssb! If I understand correctly, their approach is total decentralization, with users as nodes maintaining their own logs. But this is not what most GitHub users are used to. I wish there could be as less burden as possible for users from GitHub, which in my approach (arguably a middle ground between centralization and peer-to-peer) is as simple as:

1. sign up an account on a node
2. make a copy of their GitHub repo
3. add a remote to their local copy
4. do git stuff as usual

...while the peer-to-peer approach loads the developers with extra and currently unnecessary burden of, e.g.

• data management (a laptop is more likely to get stolen than a 19-inch rack)
• multi-device synchronization (you don't have one account for each device holding your data)

IMO the GitHub alternatives are doing really well, and implements GitHub features quite seamlessly. We should take advantage of that. And what's stopping them from overtaking GitHub? Lack of adequate userbase. A plugin-like thing would be lovely; we can use the GitHub API to intergrate GitHub with the federated servers.

Note for fed git devs: The ssb is one clever idea. That should be an option.

[ShalokShalom]

Why is bittorrent successful then? Distributed is Git already - Github is a centralised platform for a completely distributed service. Linus Torvalds said that the attribute of being distributed is the number one benefit which such a system provides. I suggest to reduce the burden, opposed to the safety net. :)

I love Gitea, that being said. I also can see Go working fine in such an environment, while somethin on Erlangs VM could be ideal.

Just me considering all the options.

[fakefred]

@ShalokShalom I agree that git is already clever itself, and decentralized as well. We're trying to replicate GitHub's social platform functionalities.

Please correct me if I make any mistakes/misunderstandings in the following paragraph.

Does ssb require a static IP address? Because if it does, afaik IPv6 coverage is quite low where I am, i.e. China, especially with all those ISPs reluctant to deploy it. And I believe devs in more regions are unable to obtain such privilege, thus making marginalized regions still marginalized.

[ShalokShalom]

I dont think secure scuttlebut requires a static IP, I would wonder myself very hard on that.

Ah yeah: And you would be also interested into TiDB and CockroachDB, two decentralised databases with the interesting implementation of an MariaDB/Postgres API, so you can simply use one of these two.

I also think we could simply all host our source code on our own, a Raspberry Pi 4 is surely enough for most projects.

[fakefred]

@ShalokShalom
That's a relief. And the DBs are neat, sounds like they fit our use case if we need such a DB network.

While some tech-savvies are for hardcore decentralization, I'm not a big fan of hosting code "all on our own", because of reasons I mentioned yesterday, and because homeservers aren't something you can get in a grocery store (although I'd be glad to get one there). RPi4s still have not started shipping here. Maybe we give users more options... what do you think?

[ShalokShalom]

Yeah, I actually think combining all these idea is feasible. It is all Git, nonetheless so we can combine different approaches, since the technical limitations in those individual concepts could be much less of an issue opposed to the custom prioritys.

This means, since integration of much different persons into this project is a goal for us, that it also means that much different prioritys will clash together, like decentralisation and complete distribution, different hosts and different implementations.

Some might love Gitlab, some Phabricator and all can name sane reasons for it. So, this is one more indication that some form of united database could come in handy.

I honestly think Github has a lot of pro's going for it, so it will likely stay. Public uploads from the bespoken blocked countries are now fully legal and the transition is easy?

So all we are talking is about the closed source repos, correct?

[fakefred]

@ShalokShalom

What's good about software is that anyone can make a fork of what they think is not good enough and make their own implementations. What's sad about software is, like virtually anything else, you're not going to have The Verge writing about your hand-craft piece of art without users.

• me, three minutes ago

Multiple options are all good. Let users pick what they want: more of a community, or more like pure hosting. Total decentralized git hosting is already being realized by git-ssb, while in the meantime I will try to turn my opinions into an option for another type of people. I'm currently considering connecting instances of self-hosted git with an API bridge that forwards issues & PR's to where the repo is hosted.

GitHub is good itself, I believe if it were hosted in e.g. Switzerland it won't do such crap this time. But even if GitHub re-enabled access to public repos of the victims of this case, their compliance to US laws is persistent, and one cannot tell what's next. What we're looking for is a place where no government can manipulate on its own. That's where decentralized DB's are really useful, yes.

As for private repos, I don't think GitHub is really the place for them, and we know it's unlikely GitHub will lift the ban.... I like them though, they're part of my CI (soon, they won't). Why GitHub gives us private repos for free is possibly the urge to promote itself after gathering enough funding from the acquisition by Microsoft. I don't see a single advantage of hosting private repos on GitHub rather than self-hosting, except for the pricing. So private repos are not in our major consideration if we are to create a decentralized GitHub alternative that provides cloud hosting, nor is it what I'm talking about right now.

[ShalokShalom]

I'm currently considering connecting instances of self-hosted git with an API bridge that forwards issues & PR's to where the repo is hosted

Yeah, I have seen that.

Another hint, regarding these databases: Phabricator works only with the Maria API, so TiDB is your only choice once you decide to support it.

Much fun and keep us on track, thanks a lot

[fakefred]

@ShalokShalom that's a lot of encouragement and useful pointers, thank you! 😊

I will get to know about the DBs, and hopefully things will go well. Phabricator is not my first target, which I think will probably be gitea instead (some of my friends use it, thus providing more feedback).

If anyone here has any advice, just drop it in.

[ShalokShalom]

Yeah, Gitea is great \o/

[gusbemacbe]

@ShalokShalom , unfortunately I can not download Gitea's GPG signature.

[fakefred]

@gusbemacbe is that a network issue?

[gusbemacbe]

I have 100MB internet bandwidth. It took many minutes to load and it gave an error:

Proxy Error

The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request GET /pks/lookup.

Reason: Error reading from remote server

[mmahmoudian]

@fakefred I read your blog post and most (not all) the comments above. Entreatingly, I was thinking about similar concept yesterday and now after reading your blog post it is more clear. I should say the idea very much sounds like how Diaspora works (haven't dive into mastodon, but I have a feeling that it shares the same general idea). All an all, this would/could work and i think it needs some serious attention from community.

the only problem that decentralization brings to mind is the acknowledgment security. Meaning that the server/node owner has the opportunity and the upper hand to change the contributions. Perhaps that can be dealt with through some sort of distributed ledger of metadata hashes in a blockchain format to ensure that.

[ShalokShalom]

@gusbemacbe I would recommend you their issues page

[fakefred]

@mmahmoudian Thanks, fellow decentralizer!

Yes, that's pretty much how Diaspora and Mastodon work, with "pods" and "servers" replaced with git hosting nodes. But I have a feeling ActivityPub isn't gonna work entirely on its own, since we've got some code-hosting requirements. You should have read my argument on the maturity of existent open source code hosting services with a web UI, and the unnecessarity to reinvent the wheel. We can figure out what we exactly need to federate them together, and make them happen. (The topic is also being discussed over there in gitea's issue page and pretty much everywhere and are overwhelmingly long.)

Security seems more like a task of git hosting services. If necessary, we can implement a blockchain-like mechanism which requires the user's PGP key to sign the commits, as in ssb.

[fakefred]

Another heads-up: Whatever we're building, they must be user-centric, for it's the users who generate their data. It's like ssb's implementation with a federated network in which each node hosts an integrated copy of all its users' data, displayed in a GitHub-like format. The data is thus only modifiable by the user themselves, while in the servers stores a backup. The user should be able to migrate/export/remove their data without any hassle, could be achieved with decentralized DB's.

[ShalokShalom]

I can also see some form of syncing between the database 'on the server-side'
and the data 'on the user side' with the combination between PouchDB and CouchDB.

This could mean that PouchDB is the database that stores the data in the browser and syncs then into the CouchDB. Of course, somebody has to implement their API into Gitea.

I know this is something pretty different and probably also something a bit more challenging:
Therefore, these two already share the same format and PWA`s are suitable for mobile phones.

[theonlyfoxy]

Latest Valid Repo:
https://notabug.org/peers/forgefed

[mmahmoudian]

On another issue, @FIREFOXCYBER posted a link that contains a very good illustrative video by @yrashk on what the general idea is. The project that is described in the video is:

https://github.com/gitchain/gitchain

The video and other related material can be found in the Kickstarter page.

The project looks unmaintained since the last commit in the github repo was 5 years ago (Jun 16, 2014) and the latest update in the Kickstarter page is for June 2018. but it might be a good foundation to build on top.

[theonlyfoxy]

There is one more similar project to GitChain but it seems to be outdated.
https://github.com/cjb/GitTorrent
Also this one worth a check:
https://github.com/elastos/Elastos

[fakefred]

As a dev relatively new to cryptography, may I ask for an explanation of the technical implementation behind the "git chain requires the PGP private key to be appendable", or the piece of code that demonstrates such technique?

[Alir3z4]

As much as "federated" and "decentralized" system looks really shiny and all, it won't solve the problem, if it was a key to such problem, we would have it by now, everything would be working on decentralized system by now, Google would bring some stuff to offload the expenses on users and not depend on ads revenue, GitHub would do the same to save tons of money. Travis-CI would do the same thing to save tons of money, everyone would do to save something and offload the hell of pain from their shoulders.

Git is decentralized, but you still have git.kernel.org, why ?

Let me give you an example; How many times you wanted to download something (legal) from torrent and you just faced with zero seeders ? Pretty sure many times.

decentralized exists as long as nodes exists, if I publish a code and my node is down, you won't be able to get my software, no matter how critical it is, you simply won't. Before you jump to conclusion that you'll make another node to fork and bring it up etc, I'd say that will be again centralizing solution. What's next ? a node that hold the most important git repos and keeping them online 24/7 ? I'm sorry, but that would requires costs, maintenance and good amount of security to make sure the codes are not being infected with something nasty.

These systems are subject to petabytes (maybe more) of bandwidth consuming, some tiny nodes in someone basement will not handle any much mbps traffic per connection unless they're paying a lot of money for their internet connection, I myself won't commit to pay more than my time, internet connection and life that I can spend with my family over keeping a stupid node online so someone other side of globe can do free-loading (I live and breath open source, it's just an example). Maybe at the end people will put your most important stuff on GitHub as mirror repos, well congrats, your just proved Github is critical and your system is just "any moment that a node will go down".

YOU NEED THE SOURCE OF TRUTH (forget about ledger, you still need a place to keep the ledger always available and ETH network wouldn't do you any favor, gas money baby!) and that's why centralized system will win over and over and over and over again.

Also where is money, there's progress and shiny stuff, bring up the beautiful and problem solver of federated system but after a while you no longer have any interest in developing it, other won't as well, unless they're paid, because anyone needs money to function, even the most famous libs that may not be 1000 LOC will get abandoned by their maintainers/devs because they have a life and need to get onto it.

Again, you need maintenance, a set of moderation system in place to keep the nasty nodes out, make sure your system it's not used for evil purposes, you need oh my god, a lot of things, just because it's decentrialized/federated it doesn't mean it's free and it will be on auto-pilot.

Well, feel free to build a federated/decentralized system the way you like while you have interests and excitement to do so, but make sure you understand the financial, survival and importance of the project.

[fakefred]

@Alir3z4 at first I wanted to respond with "sad but true".

But noooo, we won't quit the endeavor. As my blog post suggests, each host node can adopt its own monetization method. This means a sort of competition can be formed.

On second thoughts, this is, in fact, but a competition of money. If one node goes especially well, we just end up registering on and using one node eventually... Which is no different than centralization. Very bad. Ideas desperately needed.

So, sad but true.

[ShalokShalom]

Same things were said about open source

[fakefred]

@ShalokShalom thanks for the uplifting remark. I like how the git-ssb devs said in their docs that they trust fellow scuttlers, who will not "mess with other people's master branch".

Although the outcomes may not be platonic, we will have tried our best.