Skip to content

Bringing back realistic RAID recovery to inexpensive disks

License

Notifications You must be signed in to change notification settings

Fullaxx/microraids

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

microraids

Fighting BITROT and URE with every last breath

Overview

I have always been a huge proponent of typical RAID. Back in the day of 2TB drives, disks were quite reliable. Even when a disk failed, I never had a problem doing a RAID recovery/resync onto a new disk. The RAID would keep running for days or weeks no problem even with 1 disk missing, up until the point a new disk could be inserted and replace the bad disk. From that perspective, the RAID system was very reliable. Fast-Forward to today where 8TB drives are ubiquitous and the probability of doing a RAID5 recovery a 9x8TB array is between 0.3% and 56.2%. I had to learn about URE percentages the hard way, so here we are.
I propose that the fundamentals of RAID are still good (including the software), but doing a full recovery on a LARGE array is no longer a realistic option given the bit error rates of current drives (i.e. RAID5 on a 72TB array). I choose to make many small "microraids" to encapsulate my data. This will keep the recovery percentages very high for each array. Each microraid is backed by a set of disk images, placed anywhere on any disk. For each microraid you can choose a different level of redundancy, even though they are stored on the same set of physical disks. microraids gives the flexibility to have any number of RAID 0/1/4/5/6 arrays as long as available drive space will allow it. Also each microraid can be checked independently for integrity and consistency in multiple ways.

Advantages

  • Disk capacities do not have to match
  • Integrity issues can be identified before recovery
  • Recovery probabilities are significantly increased
  • You can put multiple raid types (i.e. 0/1/4/5/6) on the same disk

Required Software

  • calc
  • Standard Utilities: sgdisk / hdparm / dd / losetup / mdadm

HOWTO

Tested Operating Systems

Simple Mini-ITX Setup

Alternate Hardware

More Information

About

Bringing back realistic RAID recovery to inexpensive disks

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages