Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a script to upload data to the-federation.info #244

Open
2 tasks
Minoru opened this issue Dec 18, 2023 · 1 comment
Open
2 tasks

Add a script to upload data to the-federation.info #244

Minoru opened this issue Dec 18, 2023 · 1 comment
Labels
enhancement New feature or request

Comments

@Minoru
Copy link
Owner

Minoru commented Dec 18, 2023

I've been running the following every weekend:

./scripts/fetch_the-federation > thefed && sort --unique --output=thefed thefed && wget -O nodes.json https://nodes.fediverse.party/nodes.json && jq '' nodes.json | grep --invert-match '^[][]' | sed 's#^\s*"##; s#",*$##' > nodes && sort --unique --output=nodes nodes && diff -ruN thefed nodes | grep '^+[^+]' | sed 's#^+##' | wl-copy ; rm -f nodes nodes.json thefed

and then uploading it to the-federation.info via its mass-register Web interface. I've been meaning to automate this for a while now, but never got around to it.

Well, now the mass-register interface is broken and it won't be fixed anytime soon (https://codeberg.org/thefederationinfo/the-federation.info/issues/338), so it's high time I wrote a script -- this time using the single-register interface.

TODO:

  • write a script that compares DB to what scripts/fetch_the_federation returns and reports the difference to the-federation.info
  • Ansible: install a systemd timer to run the new script once a day
@Minoru Minoru added the enhancement New feature or request label Dec 18, 2023
@crapStone
Copy link

https://codeberg.org/thefederationinfo/next/issues/90

It is going to take a while until we have time to implement it directly into our crawler.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants