Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: LVM support #75

Open
Hurricos opened this issue Jul 27, 2022 · 1 comment
Open

feat: LVM support #75

Hurricos opened this issue Jul 27, 2022 · 1 comment

Comments

@Hurricos
Copy link
Contributor

Hi there,

I authored rlb so I can do LVM-snapshot based, mounted backups using Restic. This combines atomic backups with all the advantages of having the filesystem mounted; the backups are invoked by systemd timers and the service triggered by the timer is intentionally restricted from running when fstrim.service runs so as to avoid fstrim + dm-snapshot issues that I'd encountered a few times.

The advantage: Using lvm to create a mountable, atomic snapshot of a filesystem for backup means that all of the advantages restic gets (similar to what rsync gets) for reading not just a filesystem but the same filesystem it read previously are conferred to restic -- in particular, extremely fast and small snapshots, reading from cached pages on the underlying block device (dm-snapshot does that, it's really incredible) -- while also providing an atomic view of the snapshot.

Anyways, I'm working on implementing that here, using (in addition to what I have under rlb):

  • mount namespaces (using man 5 systemd.exec's PrivateMounts=on in the [Service] configuration part of the unit)
  • bind-mounting the subdirectory being backed up to the same directory within the mount-namespace
  • a few additional role variables describing the backup process.

Questions:

  • Are we OK restricting that functionality to systems using systemd?
  • Are we OK with the additional functionality and configuration options here?
  • Are we OK with making it clear that the filesystems this works on are restricted?
    • I'm going to guess the kernel is going to be unhappy finding out there are two nearly-identical (by UUID) BTRFS devices attached to it, for example, even when it doesn't have the same concerns with XFS or EXT4 -- so we'd probably want to restrict the use of LVM to backing-up subdirectories on those filesystems.
  • Would we prefer forcing the role administrator to be explicit with the details about the mountpath being backed up? (e.g. force them to provide the VG / LV name, how much snapshot space they want, etc).
@Hurricos
Copy link
Contributor Author

Hurricos commented Jul 27, 2022

bind-mounting the subdirectory being backed up to the same directory within the mount-namespace

This won't work for /, so perhaps instead, bind-mount somewhere so that Restic backs things up consistently (e.g. bind-mount everything to /_/).

This is needed in order to make snapshots scan consistently until restic/restic#3200 is merged.

Hurricos added a commit to Hurricos/ansible_role_restic that referenced this issue Aug 27, 2022
This commit implements the needs of roles-ansible#75[^1]: it allows for the
creation of atomic backups when the backup target is a file/dir whose
fs rests on LVM. This ensures the snapshot will be atomic.

By using a mount namespace, the LVM snapshot can be done in the same
directory -- so existing LVM-based applications of ansible_role_restic
can be migrated to this implementation without any discontinuity in
what appears to be backed up.

This combination of LVM's snapshotting layer and mount namespaces
comes with some caveats:

- you cannot backup / due to namespace issues
- subdirs with a separate fs won't be correctly detected
- not all filesystems are happy about LVM snapshots -- btrfs, e.g.
- LVM snapshots come with a performance penalty when active
- fstrim and LVM snapshots don't like each other whatsoever

[^1]: roles-ansible#75
Hurricos added a commit to Hurricos/ansible_role_restic that referenced this issue Aug 28, 2022
This commit implements the needs of roles-ansible#75[^1]: it allows for the
creation of atomic backups when the backup target is a file/dir whose
fs rests on LVM. This ensures the snapshot will be atomic.

By using a mount namespace, the LVM snapshot can be done in the same
directory -- so existing LVM-based applications of ansible_role_restic
can be migrated to this implementation without any discontinuity in
what appears to be backed up.

This combination of LVM's snapshotting layer and mount namespaces
comes with some caveats:

- you cannot backup / due to namespace issues
- subdirs with a separate fs won't be correctly detected
- not all filesystems are happy about LVM snapshots -- btrfs, e.g.
- LVM snapshots come with a performance penalty when active
- fstrim and LVM snapshots don't like each other whatsoever

[^1]: roles-ansible#75

--
Changes in v2:
- Use `findmnt -v` to find snapshot when cleaning up
- Check for _snap before `lvremove -y`

Signed-off-by: Martin Kennedy <[email protected]>
Hurricos added a commit to Hurricos/ansible_role_restic that referenced this issue Aug 28, 2022
This commit implements the needs of roles-ansible#75[^1]: it allows for the
creation of atomic backups when the backup target is a file/dir whose
fs rests on LVM. This ensures the snapshot will be atomic.

By using a mount namespace, the LVM snapshot can be done in the same
directory -- so existing LVM-based applications of ansible_role_restic
can be migrated to this implementation without any discontinuity in
what appears to be backed up.

This combination of LVM's snapshotting layer and mount namespaces
comes with some caveats:

- you cannot backup / due to namespace issues
- subdirs with a separate fs won't be correctly detected
- not all filesystems are happy about LVM snapshots -- btrfs, e.g.
- LVM snapshots come with a performance penalty when active
- fstrim and LVM snapshots don't like each other whatsoever

[^1]: roles-ansible#75

--
Changes in v2:
- Use `findmnt -v` to find snapshot when cleaning up
- Check for _snap before `lvremove -y`

Signed-off-by: Martin Kennedy <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant