Computerized Diagnosis of Liver Tumors From CT Scans Using a Deep Neural Network Approach

Abstract

The liver is a frequent site of benign and malignant, primary and metastatic tumors. Hepatocellular carcinoma (HCC) and intrahepatic cholangiocarcinoma (ICC) are the most common primary liver cancers, and colorectal liver metastasis (CRLM) is the most common secondary liver cancer. Although the imaging characteristic of these tumors is central to optimal clinical management, it relies on imaging features that are often non-specific, overlap, and are subject to inter-observer variability. Thus, in this study, we aimed to categorize liver tumors automatically from CT scans using a deep learning approach that objectively extracts discriminating features not visible to the naked eye. Specifically, we used a modified Inception v3 network-based classification model to classify HCC, ICC, CRLM, and benign tumors from pretreatment portal venous phase computed tomography (CT) scans. Using a multi-institutional dataset of 814 patients, this method achieved an overall accuracy rate of 96%, with sensitivity rates of 96%, 94%, 99%, and 86% for HCC, ICC, CRLM, and benign tumors, respectively, using an independent dataset. These results demonstrate the feasibility of the proposed computer-assisted system as a novel non-invasive diagnostic tool to classify the most common liver tumors objectively.

Fig. 1.

Block diagram of the proposed liver tumor classification scheme.

Fig. 2.

Accuracy of the training, validation, and test sets over different epochs.

Fig. 3.

t-SNE plot of features for all scans included in this study.

Fig. 4.

HCC, ICC, CRLM, and benign tumors correctly diagnosed by the neural network, as well as corresponding activation maps; The top to bottom rows are for HCC, ICC, CRLM, and benign tumors, respectively. Tumors are shown in the first, third, fifth, and sixth columns and their corresponding activation maps are shown in the second, fourth, sixth, and eighth columns. The fifth to eighth columns are for marginal cases.

Fig. 5.

HCC, ICC, CRLM, and benign tumors misdiagnosed by the neural network. The top to bottom for HCC, ICC, CRLM, and benign tumors, respectively. Left to right: HCCs are detected as ICC, CRLM, and benign tumors; ICCs are detected as HCC, CRLM, and benign tumors; CRLMs are detected as HCC, ICC, and benign tumors; Benign tumors are detected as HCC, ICC, CRLM.