Separation of bones from soft tissue in chest radiographs: Anatomy-specific orientation-frequency-specific deep neural network convolution

Abstract

Purpose: Lung nodules that are missed by radiologists as well as by computer-aided detection (CAD) systems mostly overlap with ribs and clavicles. Removing the bony structures would result in better visualization of undetectable lesions. Our purpose in this study was to develop a virtual dual-energy imaging system to separate ribs and clavicles from soft tissue in chest radiographs.

Methods: We developed a mixture of anatomy-specific, orientation-frequency-specific (ASOFS) deep neural network convolution (NNC) experts. Anatomy-specific (AS) NNC was designed to separate the bony structures from soft tissue in different lung segments. While an AS design was proposed previously under our massive-training artificial neural networks (MTANN) framework, in this work, we newly mathematically defined an AS experts model as well as its learning and inference strategies in a probabilistic deep-learning framework. In addition, in combination with our AS experts design, we newly proposed the orientation-frequency-specific (OFS) NNC models to decompose bone and soft-tissue structures into specific orientation-frequency components of different scales using a multi-resolution decomposition technique. We trained multiple NNC models, each of which is an expert for a specific orientation-frequency component in a particular anatomic segment. Perfect reconstruction discrete wavelet transform was used for OFS decomposition/reconstruction, while we introduced a soft-gating layer to merge the predictions of AS NNC experts. To train our model, we used the bone images obtained from a dual-energy system as the target (or teaching) images while the standard chest radiographs were used as the input to our model. The training, validation, and test were performed in a nested two-fold cross-validation manner.

Results: We used a database of 118 chest radiographs with pulmonary nodules to evaluate our NNC scheme. In order to evaluate our scheme, we performed quantitative and qualitative evaluation of the predicted bone and soft-tissue images from our model as well as the ones of a state-of-the-art technique where the "gold-standard" dual-energy bone and soft-tissue images were used as the reference images. Both quantitative and qualitative evaluations demonstrated that our ASOFS NNC was superior to the state-of-the-art bone-suppression technique. Particularly, our scheme was better able to maintain the conspicuity of nodules and lung vessels, comparing to the reference technique, while it separated ribs and clavicles from soft tissue. Comparing to a state-of-the-art bone suppression technique, our bone images had substantially higher (t-test; P < 0.01) similarity, in terms of structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR), to the "gold-standard" dual-energy bone images.

Conclusions: Our deep ASOFS NNC scheme can decompose chest radiographs into their bone and soft-tissue images accurately, offering the improved conspicuity of lung nodules and vessels, and therefore would be useful for radiologists as well as CAD systems in detecting lung nodules in chest radiographs.

Keywords: bone suppression; chest x-ray; deep learning; neural network.

Figure 1

Schematic diagram of our virtual DE system based on our deep ASOFS NNC model in a test stage. The original unseen single CXR is decomposed into specific orientation‐frequency components in multiple anatomic segments. The decomposed components are selectively assigned to corresponding trained NNC experts by the gating layer. The resulting bone predictions from multiple NNC experts are merged by the soft‐gating layer followed by the OFS reconstruction to form a complete bone image where soft‐tissue components are removed, while bone components are maintained. [Color figure can be viewed at wileyonlinelibrary.com]

Figure 2

Detailed schematic of our AS NNC with gating and soft‐gating layer. Each NNC is trained through its corresponding patch‐pixel pairs in a specific lung segment. The predictions are combined through a soft‐gating layer by a weighted combination of per‐segment predictions. [Color figure can be viewed at wileyonlinelibrary.com]

Figure 3

Detailed schematic of our OFS NNC with OFS decomposition/reconstruction. From a deep‐learning perspective, each decomposition corresponds to convolution followed by a pooling layer. Similarly, reconstruction is equivalent to an up‐sampling layer followed by convolution. Each NNC is trained individually through its corresponding patch‐pixel pairs of a specific orientation‐frequency component. [Color figure can be viewed at wileyonlinelibrary.com]

Figure 4

Pair‐wise comparison of our ASOFS NNC scheme with the reference‐standard AS MTANN technique in terms of (a) SSIM and (b) PSNR for both bone and soft‐tissue images. The difference metrics between the two techniques were sorted in an ascending order. Note that the x‐axis is the sorted image index and does not necessarily correspond to the image index. [Color figure can be viewed at wileyonlinelibrary.com]

Figure 5

Qualitative comparison of (c) our ASOFS NNC scheme with (b) the reference‐standard AS MTANN for two cases. A region‐of‐interest with a nodule is enlarged in each case for visual assessment. The (a) original CXRs and (d) “real” DE soft‐tissue images from our DE database are shown as references. [Color figure can be viewed at wileyonlinelibrary.com]

Figure 6

Qualitative assessment of the bone images produced by our (b) ASOFS NNC scheme. (a) The original CXR image and (c) the “real” DE bone image from our DE database are shown as references. Note that the bone images that we used for training were preprocessed in order to reduce the effect of noise of the bone images from the DE system.

Figure 7

Component‐wise bone predictions using our (b) OFS mixture‐of‐experts deep NNC scheme. Our scheme was able to successfully convert different orientation‐frequency components of (a) the original CXR into its corresponding bone components, which was similar to (c) the wavelet decomposition of the corresponding “real” DE bone image from our DE database.

Figure 8

Comparison of (a) two‐level and (b) three‐level OFS decomposition in terms of the bone images using our ASOFS NNC schemes.

Figure 9

Predictions of the AS NNC experts for different lung segments for the lowest resolution/frequency. Each NNC was trained using the patch‐pixel pairs of its corresponding segment.