What happened to this project?

[Zero3K]

subject

[jgottula]

I started (and stopped) working on this project about a decade ago, back when I had basically no idea what I was doing.

I subsequently realized that I had no idea what I was doing, and so I stopped working on it.

That's basically the story. I decided to keep the repo up, in case anyone might find it interesting or useful; but I tried to keep expectations suitably low, given that I had no idea what I was doing and a lot of stuff was not particularly well-implemented.

If I remember right, my B-tree traversal code was, uh... not particularly optimal nor good.

In any case, I think there are other, more professionally written Btrfs-for-Windows projects out there these days; I don't keep up on that anymore though.

---

Btrfs has happily eaten my data several times over the years (including recently in its """stable""" status), and as a result I've lost all respect for Btrfs as anything but an Ext4 stand-in that happens to have a couple of convenient features like compression and reflinks.

I wouldn't dare ever store large amounts of data on Btrfs and actually trust it to do the job well and not encounter some kind of idiotic code error that corrupts everything.

I've had a far, far better experience keeping my main data on ZFS for the majority of the time since I worked on this project. (Talking more about Linux and not so much Windows now.)

It seems to me that the OpenZFS/ZFSonLinux developers have done a much, much better job of building something that actually lives up to its promised functionality and doesn't randomly eat your data, compared to what the mainline Btrfs developers have managed to come up with over the last more-than-a-decade.

It never ceases to amaze me, the sorts of problems that Btrfs still has, given the amount of time that project has been in development. All those awesome data resiliency features, and yet somehow I've lost far more data to Btrfs than I ever have to Ext4...

[Zero3K]

1. There's only one that is open source. It is https://github.com/maharmstone/btrfs/. It has some issues though (such as having crashes and issues with showing the actual amount of free space left).
2. The developer has a huge number of issues that are still open.
3. The developer doesn't like me reporting issues its repository.
4. I haven't had issues with data corruption caused by using BTRFS when using it with ReactOS.

[jgottula]

I see.

Filesystem interop between OSes is so much worse than it ought to be. The fact that FAT32 is still, to this day, generally agreed upon as the best you can reliably and easily use across the major platforms (i.e. Windows/Linux/macOS), is ridiculous. I guess NTFS is arguably the superior option, as both Linux and macOS have developed pretty good drivers for it. But still though... it's idiotic that the situation is the way it is.

---

To be honest, even if I had sufficient interest and motivation to continue work on this particular project, I would almost certainly want to start over from scratch.

I suspect that a better approach to this project might have been to reuse the upstream Btrfs code from the Linux kernel as much as possible, and implement just enough shims/wrapper code to hook up all the outward-facing interfaces (and inward-facing kernel infrastructure stuff). The way I actually went about things was to more-or-less reimplement everything myself. And that was great, from the educational aspect of forcing me to learn about how all the important data structures worked directly.

In the long run though, if trying to make a more feature-complete implementation, and especially one that would have write support and which could relatively easily stay at parity with upstream changes and improvements, the "wrapper-around-the-Linux-driver-code" approach would at least seem to make far more sense.

(Aside from various general advantages of that method, it's also unfortunately difficult to find documentation anywhere on the Btrfs wiki or elsewhere that is actually up-to-date and not decade-old partially-written stuff. So while the fundamental data structures at the core of Btrfs are pretty well documented and therefore easy to reimplement independently, a lot of the less-fundamental parts of the filesystem are effectively just not documented at all, aside from "read the code and figure out how it works". Which would ostensibly put the "wrapper" approach at a further advantage.)

But then again, properly implementing all of Btrfs is inevitably a giant pain in the ass no matter how you approach it, particularly because of features inherent to its design, such as its multi-device support. Besides just kind of being obnoxious in the first place (see note 1 below for my thoughts on that), is self-admittedly a sort of "rampant layering violation" that basically forces any other-OS implementation to handle all that "clever" device management junk itself, which is especially annoying in most traditional contexts (like Windows) where there's sort of a base-level assumption of a 1:1 mapping between a block device and the filesystem on it.

And actually... thinking about it a bit more... the upstream Btrfs project has always been about making a filesystem driver as part of the Linux kernel (never any serious concerns about portability), so it probably has tons and tons of Linux-kernel-specific infrastructure dependencies inherent to its implementation that'd be a nightmare to ever "just shim/wrapper" your way out of.

Basically, everything sucks. 😕

---

There's only one that is open source. It is https://github.com/maharmstone/btrfs/.

I took a quick look at this project just now. And at least as far as the list of implemented features, and the manner in which it's been done (as an actual native kernel driver, rather than the FUSE-like approach I went with, based on the long-ago-defunct-but-sort-of-revived-since-then Dokan library; see note 2), it's really quite impressive to me. On top of the fact that it's also a reimplementation rather than a wrapper or anything of that sort. And ReactOS compatibility is definitely cool to see as well.

So to be honest, it sounds like he's managed to accomplish more-or-less what I would have liked to be able to do myself originally (that I simply lacked the continued interest and motivation to ever get even close to finishing).

It has some issues though (such as having crashes and issues with showing the actual amount of free space left).
The developer has a huge number of issues that are still open.
The developer doesn't like me reporting issues its repository.

It's definitely a shame to hear that you haven't had very good experiences trying to get issues resolved with that developer. And that there are lots of unresolved issues as well as crash problems. (To be honest, the fact that the thing even apparently works at all, in spite of the problems it evidently does have, is still pretty impressive to me; and it doesn't surprise me that it would still have a ton of issues that would be a huge pain to work out.)

As a sort-of-defense of that project's developer apparently not being great about being responsive to issues... and maybe as a bit of humor... I will say that I can only imagine how horrendously awful it must have been to subject oneself to writing a very-feature-compatible, independent implementation of Btrfs for Windows (as a kernel driver, no less); and so to some extent I have to imagine that if I were in that position myself, the process of coding that thing would probably have turned me into an extremely grumpy old man inside. 😝

---

I haven't had issues with data corruption caused by using BTRFS when using it with ReactOS.

Just to be clarify here, my complaints about Btrfs and data corruption come from my experiences over the years using the original implementation in the Linux kernel. Something about the way the Linux Btrfs devs chose to develop their filesystem and the Linux code for it just seems not-well-thought-out and problematic.

And you got idiocy like their RAID56 functionality still being "WARNING THIS DOES NOT ACTUALLY WORK AND WILL EAT ALL OF YOUR DATA SO DON'T ACTUALLY USE IT" so far into the filesystem's development (IIRC that was only recently fixed); and more idiocy like "I guess we do need to have an fsck tool because it turns out that stuff gets corrupted all the time; but we'll write the official documentation so it says to never use it, and that it is likely to just ruin your data more, and that literally you should contact the developers to figure out what to do before you take any actions"... Yeah. I used to be excited about Btrfs back in the 2010-2012 timeframe, but the more problems I experienced and the more "why is this still a problem after X years of development" situations that came up, the less interested I became.

---

Notes

1. For example: even on Linux, you cannot safely have a Btrfs filesystem mounted and then also simultaneously mount (or even have present as a device node, really, thanks to Btrfs's "helpful" device autodetection udev rules) a backup image of the same filesystem from a different point in time; because Btrfs "helpfully" identifies the two devices as both being part of the same FS due to the FS UUID being the same in literally every node of the entire tree, and it will either try to automatically put the devices together assuming they are a multi-device FS, or worse, will arbitrarily start writing to the wrong device node because it only uses the FS UUID for identification of which device node is which, which is sort of nuts. (And I guess the counter argument to that is probably "nobody needs backup images with Btrfs, because snapshots!" But in my experience, primarily using Btrfs only as a single-device Ext4 replacement for things like root volumes and such nowadays, having disk image backups is essential, and the way Btrfs deals with this stuff is nothing but a giant nuisance at best and a massive corruption risk at worst.)

2. Way back a decade ago, I initially planned to do my project as a "proper" in-kernel FS driver. I got ahold of a copy of Windows NT Filesystem Internals (which remains on my bookshelf to this day) and did my best to find other supporting documentation. But at least at the time (perhaps the situation has improved since), it just seemed excessively difficult to find good documentation with actual example code, on MSDN or otherwise, for writing true FS kernel drivers. There was a decent amount of stuff on writing FS filter drivers; but I guess it was a sufficiently rare thing for Windows software developers to actually want to write a bespoke FS kernel driver, for there to be much out there to guide me through the process. So when I discovered that a FUSE equivalent for Windows did exist, I just jumped over to that and didn't really look back (despite the many downsides).

[Zero3K]

I would be fine with someone making a fork of Mark Harmstone's BTRFS driver and fixing the crashes, etc. that he won't fix.