Rewrite Geoblock (use new mmdb)

[enoch85]

@szaimen @aaaskew Please test this PR. Please note it requires some new functions, please see the lib file in this PR.

FIx #2674

[enoch85]

This is now fully tested from me. Will merge later today, would be nice with some further testing though. Please dig in @szaimen @aaaskew 😄

[aaaskew]

Many thanks for the change. Unfortunately I can't update to the new module. I get 'Update limit for Maxmind GeoDatabase reached! Please try again tomorrow.'.

I found the following: From March 2024; Every account is limited to 1,000 total direct downloads (30 for GeoLite accounts) in a 24-hour period. (https://support.maxmind.com/hc/en-us/articles/4408216129947-Download-and-Update-Databases)

This appears to come from lib.sh and I assume this is due to the hard-coding of an Account ID and license. This means that anyone will be using your credentials to download the database.

The download appears to be done with a Docker image? Not too sure about how this works but do we need to download the City IP database? I thought we are only using the country database:

echo "GEOIPUPDATE_EDITION_IDS=GeoLite2-City GeoLite2-Country"

I believe part of the reason for MaxMind deprecating the old database format is they could not control or monetize it for corporate users with the old APIs.

I have created my own login and can download the database myself. Maybe a few more instructions would be good it add to explain that if the download fails, the database could be sideloaded? I would think if the database already exists in the correct location then the update would not fail the download. People can create their own MaxMind account and license so that the downloads will be per deployment and therefore no longer hit a limit in lib.sh.

It will make the use of this module more of a pain to setup but I think it is the only realistic way to make sure it works for everyone.

[enoch85]

Thanks for your input, I have now disabled the City DB. :) You might be correct that that's part of the problem as that takes one extra download.

Regarding license, yes, it's the only way. You need an account. Anyone will be able to abuse it, but I hope they don't. To limit the attempts I chose to save the mmdb files instead of getting new ones every time. Now in the beginning of this merge, it will take a few days to push out the new DB, but ones the dust has settled I hope that we won't be rate limited as frequently.

The mmdb files ends up in /usr/share/GeoIP and you could create your own account and fetch put them there, or you just wait. :) Eventually it will work.

Another option I've thought about is to download some old files to this repo, and use them initially, then when the user updates it will update those files. But I don't know yet, we'll see how this pans out.

[aaaskew]

Something i have seen when re-running the GeoBlock script from the menu on my recently freshed installed Nextcloud 29.

I can see code to remove the old GeoBlock stuff from apache2.conf but it looks as if it has not done so? If I look at the directory I see old and new files present.

[enoch85]

Wierd, what happens if you run this manually?

    if grep "Geoip-block-start" /etc/apache2/apache2.conf
    then
        sed -i "/^#Geoip-block-start/,/^#Geoip-block-end/d" /etc/apache2/apache2.conf
    fi

[aaaskew]

That works but is the logic correct earlier.

I assume that line 26 of geoblock.sh should be saying 'If old geoblock is not installed AND new geoblock is not installed' then this is a fresh install, otherwise, uninstall?

[enoch85]

If old geoblock is not installed AND new geoblock is not installed' then this is a fresh install, otherwise, uninstall?

Well, good point. You're absolutely right!

[abdullahdevrel]

I work for IPinfo, where we have a free IP to Country database with more lenient licensing terms and better accuracy.

This ticket has been merged and closed, so I have opened a new issue ticket pitching our free IP to Country database with more lenient license terms, which are particularly helpful for packaging and distributing data.

Our database is licensed under CC-BY-SA 4.0, which permits distribution (even for commercial purposes). We only require an attribution statement. This is one aspect of the database there is more. Please share your thoughts on my feature request ticket: #2682