expire binlog #2350
Open
expire binlog #2350
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
HPPinata
force-pushed
the
binlog-fix
branch
2 times, most recently
from
46db50a to
8f2cd07
Compare
expire binlog entries older than a week to avoid unbounded binlog growth over longer timespans Signed-off-by: HPPinata <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Enabeling binlog without any cleanup/purge policy is a horrible idea, since it causes
binlog.XXXXXXfiles to accumulate indefinitely, unless the administrator manually runs a purge by connecting to the database.I only noticed this due to the fact almost
50GBof binlog files had accumulated on my server over a period of around 7 months, even on my relatively small instance.I'm unsure why the binlog is enabled in the first place, since none of these setups are using replication, but the ability to do a point in time recovery might be the original intent.
Whatever the reasoning for enabeling binlog, this change should both stop new deployments from running into the issue and resolve the problem on any existing setups once the
compose.yamlis updated.The only exceptions are database servers under souch heavy load that the binlog grows quickly enough to reach a problematic size in the 7 days until it is cleaned up. The administrators of those instances (if they aren't already supported by Nextcloud Enterprise) would have noticed this issue and should be equipped to independently evaluate the usage of binlog on their database system.
In any case (and even for systems under high load) automatically purging binlog entries older than a week should have a significant positive impact on the deployments storage usage without negatively affecting either replication (if configured) or point in time recovery.
--slave_connections_needed_for_purge=0is required to make the binlog cleanup on databases without replication possible, which should apply to essentially all deployments based on thesecompose.yamlfiles.Including a check for a binlog without any cleanup active in the Web Ui's "security & setup warnings" would probably also be a good idea, but that's beyond my ability and should probably be a separate Issue/PR on https://github.com/nextcloud/server.
References:
https://mariadb.com/kb/en/using-and-maintaining-the-binary-log/
https://mariadb.com/kb/en/replication-and-binary-log-system-variables/#expire_logs_days