Desmoke

Multiscale deep desmoking for laparoscopic surgery

Abstract

In minimally invasive surgery, smoke generated by such as electrocautery and laser ablation deteriorates image quality severely. This creates discomfortable view for the surgeon which may increase surgical risk and degrade the performance of computer assisted surgery algorithms such as segmentation, reconstruction, tracking, etc. Therefore, real-time smoke removal is required to keep a clear field of view. In this paper, we propose a real-time smoke removal approach based on Convolutional Neural Network (CNN). An encoder-decoder architecture with Laplacian image pyramid decomposition input strategy is proposed. This is an end-to-end network which takes the smoke image and its Laplacian image pyramid decomposition as inputs, and outputs a smoke free image directly without relying on any physical models or estimation of intermediate parameters. This design can be further embedded to deep learning based follow-up image guided surgery processes such as segmentation and tracking tasks easily. A dataset with synthetic smoke images generated from Blender and Adobe Photoshop is employed for training the network. The result is evaluated quantitatively on synthetic images and qualitatively on a laparoscopic dataset degraded with real smoke. Our proposed method can eliminate smoke effectively while preserving the original colors and reaches 26 fps for a video of size 512 × 512 on our training machine. The obtained results not only demonstrate the efficiency and effectiveness of the proposed CNN structure, but also prove the potency of training the network on synthetic dataset.

Sample Result

Network

How to train

• To train the network Desmoke.ipynb

Training data folder structure is

• MainDir
  • SmokeFreeImage1
    • Image0001.png
• MainDir
  • SmokeHeavy
    • Image0001_1.png
  • SmokeMedi
    • Image001_1.png
  • etc

Testing

• Place the input image containing smoke in

filepath='./Input/'

• and Run

main_run.py

• The result will be saved in

pred_dir='./Desmoked'

Reference

If you find this code useful please cite

Wang, Congcong, et al. "Multiscale deep desmoking for laparoscopic surgery." Medical Imaging 2019: Image Processing. Vol. 10949. International Society for Optics and Photonics, 2019.

@inproceedings{wang2019multiscale, title={Multiscale deep desmoking for laparoscopic surgery}, author={Wang, Congcong and Mohammed, Ahmed Kedir and Cheikh, Faouzi Alaya and Beghdadi, Azeddine and Elle, Ole Jacob}, booktitle={Medical Imaging 2019: Image Processing}, volume={10949}, pages={109491Y}, year={2019}, organization={International Society for Optics and Photonics} }