[6.12] Track btrfs patches #36
Conversation
Add the following flags to give an hint about which chunk should be allocated in which a disk. The following flags are created: - BTRFS_DEV_ALLOCATION_PREFERRED_DATA preferred data chunk, but metadata chunk allowed - BTRFS_DEV_ALLOCATION_PREFERRED_METADATA preferred metadata chunk, but data chunk allowed - BTRFS_DEV_ALLOCATION_METADATA_ONLY only metadata chunk allowed - BTRFS_DEV_ALLOCATION_DATA_ONLY only data chunk allowed Signed-off-by: Goffredo Baroncelli <[email protected]>
Signed-off-by: Goffredo Baroncelli <[email protected]>
Signed-off-by: Kai Krakow <[email protected]>
When this mode is enabled, the chunk allocation policy is modified as follow. Each disk may have a different tag: - BTRFS_DEV_ALLOCATION_PREFERRED_METADATA - BTRFS_DEV_ALLOCATION_METADATA_ONLY - BTRFS_DEV_ALLOCATION_DATA_ONLY - BTRFS_DEV_ALLOCATION_PREFERRED_DATA (default) Where: - ALLOCATION_PREFERRED_X means that it is preferred to use this disk for the X chunk type (the other type may be allowed when the space is low) - ALLOCATION_X_ONLY means that it is used *only* for the X chunk type. This means also that it is a preferred choice. Each time the allocator allocates a chunk of type X , first it takes the disks tagged as ALLOCATION_X_ONLY or ALLOCATION_PREFERRED_X; if the space is not enough, it uses also the disks tagged as ALLOCATION_METADATA_ONLY; if the space is not enough, it uses also the other disks, with the exception of the one marked as ALLOCATION_PREFERRED_Y, where Y the other type of chunk (i.e. not X). Signed-off-by: Goffredo Baroncelli <[email protected]>
This is useful where you want to prevent new allocations of chunks on a disk which is going to removed from the pool anyways, e.g. due to bad blocks or because it's slow. Signed-off-by: Kai Krakow <[email protected]>
This is useful where you want to prevent new allocations of chunks to a set of multiple disks which are going to be removed from the pool. This acts as a multiple `btrfs dev remove` on steroids that can remove multiple disks in parallel without moving data to disks which would be removed in the next round. In such cases, it will avoid moving the same data multiple times, and thus avoid placing it on potentially bad disks. Thanks to @Zygo for the explanation and suggestion. Link: kdave/btrfs-progs#907 (comment) Signed-off-by: Kai Krakow <[email protected]>
|
Hi. What's the status of these patches? Are these something that's going be upstream in a reasonable amount of time or a long-term external patch series? |
|
These won't go in into the kernel as-is and may be replaced by some different implementation in the kernel sooner or later. But I keep those safe to use - aka they don't create incompatibilities with future kernels and can just be dropped from your kernel without posing any danger to your btrfs. @Forza-tng has some explanations why those patches won't go into the kernel: https://wiki.tnonline.net/w/Btrfs/Allocator_Hints |
|
Hi. I noticed that type 0 The result for this RAID10 was that new data were stored only on the new disks (ID 15 and 16) with a 2 stripe RAID10 instead of the expected 10 stripe RAID10. While this was unintended, perhaps this effect could be used to tier data chunks? To test this, I created a 3 device btrfs: 1 = ssd We can see that the smaller OK, perhaps not very useful since we cannot easily move hot data back on the ssd. However, it could suffice as an emergency overflow to avoid ENOSPC if your workload can survive the reduced iops... :) |
|
Yes, I think this is intentional behavior of the initial version of the patches: The type numbers are generally used as a priority sort with the I'm not sure if it would be useful to put data on I think my idea of using chunk size classes for tiering may be more useful than this side-effect (what I mentioned in a report over at btrfs-todo). But in theory, type 0 and type 3 should be treated equally as soon as the remaining unallocated space is identical... Did you reach that point? (looks like your loop dev example did exactly that if I followed correctly) But in the end: Well, "preferred" means "preferred", doesn't it? ;-) |
The loop test did the opposite of this, where the type 0 device was filled before the type 3, even though it was smaller/had less unallocated. I had expected types 3 and 0 were treated equally, but we see that this isn't the case? It isn't wrong or bad, just something I hadn't thought would happen.
Indeed 😁 |
Export patch series: https://github.com/kakra/linux/pull/36.patch
Here's a good guide by @Forza-tng: https://wiki.tnonline.net/w/Btrfs/Allocator_Hints. Please leave them a nice comment. Thanks. :-)
To make use of the allocator hints, add these to your kernel. Then run
btrfs device usage /path/to/btrfsand take note of which device IDs are SSDs and which are HDDs.Go to
/sys/fs/btrfs/BTRFS-UUID/devinfoand run:echo 0 | sudo tee HDD-ID/typeto prefer writing data to this device (btrfs will then prefer allocating data chunks from this device before considering other devices) - recommended for HDDs, set by defaultecho 1 | sudo tee SSD-ID/typeto prefer writing meta-data to this device (btrfs will then prefer allocating meta-data chunks from this device before considering other devices) - recommended for SSDsecho 4 | sudo tee LEGACY-ID/typeecho 5 | sudo tee LEGACY-ID/typeImportant note: This recommends to use at least two independent SSDs so btrfs meta-data raid1 requirement is still satisfied. You can, however, create two partitions on the same SSD but then it's no longer protected against hardware faults, it's essentially dup-quality meta-data then, not raid1. Before sizing the partitions, look at
btrfs device usageto find the amount of meta-data, at least double that size to size your meta-data partitions.This can be combined with bcache by directly using meta-data partitions as a native SSD partition for btrfs, and only using data partitions routed through bcache. This also takes a lot of meta-data pressure from bcache, making it more efficient and less write-wearing as a result.
Real-world example
In this example,
sdeis a 1 TB SSD having two meta-data partitions (2x 128 GB) with the remaining space dedicated to a single bcache partition attached to my btrfs pool devices:A curious reader may find that
sde1andsde3are missing, which is my EFI boot partition (sde1) and swap space (sde3).