LNHG model for lung nodule segmentation and intra-nodular heterogeneity image generation

This is the repository of LNHG model for end-to-end lung nodule segmentation and intra-nodular heterogeneity image generation.

Citation: Song, Jiangdian, et al. "Automatic lung nodule segmentation and intra-nodular heterogeneity image generation." IEEE Journal of Biomedical and Health Informatics (2021).

Segmentation branch using WGAN-GP for lung nodule segmentation and intra-nodular heterogeneity map production

Input: Volumes consisted of continuous CT images of the same nodule (Channel, Depth, H, W)
Output: The corresponding lung nodule images (Channel, Depth, H, W)

• Train.py is used to start the training of the WGAN-GP branch.
  
• Unet_Comparison.py is the model of the U-net for comparison.

False-positive reduction branch using Faster R-CNN with GIoU loss

Input: Original CT images (H, W)
Output: CT images of corresponding nodule candidates (H, W)

• train.py is used to start the training and validation of the Faster R-CNN branch.
  
• The _smooth_l1_loss() and the GIoU loss are provided in the same function to calculate the location loss for better understanding.
  

To explain the workflow of the proposed LNHG model more clearly, a detailed description of the procedure of lung nodule image generation is shown in Figs. F1–F3. The figures below elaborated the segmentation of lung nodules, production of the final lung nodule image, and intra-nodular heterogeneous conversion from the produced lung nodule image.

Supp_S1

Fig. F1. Explanation of the Workflow of the Proposed LNHG Model. The input is the original CT image, and Block1–Block8 denote the network layers (with the same color) presented in Figure 3 in the manuscript. The feature map output by each block showed in the figure is the mean of all the feature maps.

Fig. F2. Example illustrating how the final lung nodule image is produced based on the output of WGAN-GP and Faster R-CNN branches. The false-positive candidates produced by the WGAN-GP branch is marked in a red oval for better display. By fusing the output of the two branches, the false-positive candidates are eliminated, and the final lung nodule image is produced.

Fig. F3. Example illustrating how the intra-nodular heterogeneity images are generated from the output of the LNHG model. Three nodules are presented: (a) original CT images, (b) output of the LNHG model, and (c) intra-nodular heterogeneity images converted from the output of the LNHG model. The original CT image is superimposed on the output image of the LNHG model. Therefore, the original CT image is shown on (c), except for the generated nodule area. The images in (c) are used for the reader study.

Supp_S3

Illustration of the intra-nodular heterogeneity images. Intra-nodular heterogeneous activity is presented, and the other area is displayed in the original CT image. The images in the first row denote the pathologically confirmed benign nodules, and the second row shows lung nodules diagnosed as malignant by pathology. Significant difference on intra-nodular heterogeneous activity between the benign and malignant lung nodules could be found according to the images generated by the proposed LNHG model.