3D watershed

[rorynolan]

Hi @aoles
I'd like watershed to work in 3D (on z-stacks). Right now I can run the function on a stack, but it seems to just perform watershed on each slice individually, rather than really recognizing 3D objects. I think I know how to implement this. Are you interested in receiving a pull-request for this? If so, I'll let you know how I think it should work.
Thanks as always,
Rory

[aoles]

Hi Rory,
thank you for reaching out to me! You're right, the current implementation performs watershed on each 2D frame individually. I would be very interested in learning your approach to 3D watershed. Looking forward to hearing from you.
Cheers,
Andrzej

[rorynolan]

Hi Andrzej,
In 3D, the analogy of using water to fill valleys doesn't really work, but the idea can be generalised as follows (I presume that objects of interest are high intensity and that low intensities have been thresholded away (to 0 or NA)).
Don't invert the image, just "fill in" pixels of high intensity (highest first) until another object or background is met.
What do you think?
Rory

[aoles]

Hi Rory,
thanks for your reply. I'm not sure I understand how your approach is different from the regular watershed. It shouldn't in principle matter whether one fills in the "holes" after inverting the image, or just goes the other way round from high to low intensities on the original image. Could you maybe clarify how this would help with 3D segmentation?
Cheers,
Andrzej

[rorynolan]

Hi Andrzej,
You're totally right. In the 2D case, inverting the image helps with visualisation in the sense that it gives you "valleys" to fill in. Here, I don't bother with the inversion since it doesn't aid visualisation.
You're right that my approach is not very different to the 2D watershed, its just a generalisation to 3D whereby now each pixel has 6 neighbours instead of 4 (now a pixel has neighbours above and below too, thereby allowing the same object to span multiple planes).
Am I making sense?
Rory

[aoles]

Thanks for the clarification! I agree, the inversion is just an implementation detail which doesn't really matter for the resulting segmentation. You're right, for a 3D watershed you would look at the 6 neighboring pixels in the x, y, and z directions.

Are you interested in implementing this? You would probably need to modify the original watershed C function to look around over the whole stack. Ideally, this behavior could be toggled by a function argument to allow 2D watershed on each frame separately as it is now, e.g., for timelapse images, and switch to the 3D mode only for true XYZ stacks.

Looking forward to hearing from you.

Cheers,
Andrzej

[rorynolan]

I am interested in implementing it, I think it would be a very cool thing to have in R. I just took a look at the watershed.cpp file and some of it is a bit beyond me. I've written Rcpp before but I've never worked with .Call() and SEXP. Can you recommend a good way for me to learn about it? If it won't take me too much time I'd do it.
Thanks,
Rory

[aoles]

Thanks Rory! It would be really awesome if you could contribute some of your skills and time to improve the package. There is a good introduction to R's C interface by Hadley Wickham. The existing watershed function already contains all the tedious type conversions to get you started. Don't hesitate to contact me if you have any further questions. I will be happy to discuss with you any intermediate implementations, or provide advise in case you get stuck.

Cheers,
Andrzej

[rorynolan]

OK thanks.
I have a bit of time at the moment, but not a lot of time. I'll work on this until the end of the week and if I'm getting somewhere, I'll persevere, but if not, I'll probably give up. I'll go ahead and fork the repo now!
Thanks,
Rory

[aoles]

Great! Looking forward to hearing about the progress.
Cheers,
Andrzej

[rorynolan]

Hi Andrzej,
After a day of hacking around I'm not really getting anywhere. I think this is a good reason for me to properly learn C++. I'll do that in my spare time and see where I get but I won't be implementing 3D watershed in the next few weeks I'm afraid. I've found this functionality in ImageJ so for now I'll use that. I'd love to see it in EBImage and I'll come back to it if I can.
Rory

[aoles]

Hi Rory,
thanks for getting back to me, and most importantly thank you for your time and efforts! The extension of the watershed algorithm to 3D shouldn't be overly complicated once you understand the current implementation. I'm looking forward hearing from you once you fell to follow up on this.
Cheers,
Andrzej