Semisupervised Training of a Brain MRI Tumor Detection Model Using Mined Annotations

Abstract

Background Artificial intelligence (AI) applications for cancer imaging conceptually begin with automated tumor detection, which can provide the foundation for downstream AI tasks. However, supervised training requires many image annotations, and performing dedicated post hoc image labeling is burdensome and costly. Purpose To investigate whether clinically generated image annotations can be data mined from the picture archiving and communication system (PACS), automatically curated, and used for semisupervised training of a brain MRI tumor detection model. Materials and Methods In this retrospective study, the cancer center PACS was mined for brain MRI scans acquired between January 2012 and December 2017 and included all annotated axial T1 postcontrast images. Line annotations were converted to boxes, excluding boxes shorter than 1 cm or longer than 7 cm. The resulting boxes were used for supervised training of object detection models using RetinaNet and Mask region-based convolutional neural network (R-CNN) architectures. The best-performing model trained from the mined data set was used to detect unannotated tumors on training images themselves (self-labeling), automatically correcting many of the missing labels. After self-labeling, new models were trained using this expanded data set. Models were scored for precision, recall, and F₁ using a held-out test data set comprising 754 manually labeled images from 100 patients (403 intra-axial and 56 extra-axial enhancing tumors). Model F₁ scores were compared using bootstrap resampling. Results The PACS query extracted 31 150 line annotations, yielding 11 880 boxes that met inclusion criteria. This mined data set was used to train models, yielding F₁ scores of 0.886 for RetinaNet and 0.908 for Mask R-CNN. Self-labeling added 18 562 training boxes, improving model F₁ scores to 0.935 (P < .001) and 0.954 (P < .001), respectively. Conclusion The application of semisupervised learning to mined image annotations significantly improved tumor detection performance, achieving an excellent F₁ score of 0.954. This development pipeline can be extended for other imaging modalities, repurposing unused data silos to potentially enable automated tumor detection across radiologic modalities. © RSNA, 2022 Online supplemental material is available for this article.

Graphical abstract

Figure 1:

Flow diagram shows data inclusion and exclusion criteria for the baseline training and test data sets. Additional image annotations were subsequently added to the training data set via the self-labeling method, as described. T1C+ = postcontrast T1-weighted images.

Figure 2:

Example of the automated line annotation to bounding box transformation. (A) The cystic and solid high-grade glioma is annotated with two line measurements mined from the picture archiving and communication system. (B) The curation pipeline geometrically squares the line annotations to generate two overlapping bounding boxes. (C) These, in turn, are converted to a single bounding box encompassing the entire lesion.

Figure 3:

(A) Schematic drawing of the semisupervised learning (self-labeling) approach to automated tumor detection. The baseline model trained from noisy incompletely labeled mined images is used to identify unlabeled tumors on an expanded training image set containing 213 652 axial postcontrast T1-weighted images. After this cycle, a new detection model is trained from scratch using the expanded bounding box training data set, which is used to identify additional unlabeled tumors on the training image set. Twelve self-labeling cycles were completed, increasing the number of training bounding boxes from 10 623 to 29 185. (B) Example training bounding boxes added during self-labeling process. Blue boxes represent new lesion annotations added during self-labeling. Green boxes are baseline training annotations automatically generated from line annotations mined from the picture archiving and communication system.

Figure 4:

(A) Precision-recall and (B) free-response receiver operating characteristic curves for each detection model. AUC = area under the receiver operating characteristic curve, R-CNN = region-based convolutional neural network.

Figure 5:

Representative false-positive and false-negative predictions from the best-performing Mask region-based convolutional neural network detection model trained using the self-labeling expanded training data set. (A) False-positive predictions included choroid plexus (top left and top right), superior sagittal sinus imaged in cross section (bottom right), small vessel loops (top middle), petrous apex (bottom left), and apparent artifact within the mobile tongue (bottom middle). (B) False-negative predictions included extra-axial tumors (top right, bottom left, and bottom middle), probably reflecting a relatively limited representation of these types of lesions within the training data set, and faintly enhancing (top left) and leptomeningeal (top middle and bottom right) lesions. Solid boxes represent ground truth, and dashed boxes represent model predictions. White solid boxes are correctly detected ground truth boxes. Red solid boxes are for false-negative predictions. Red dashed boxes are for false-positive predictions. Dashed blue boxes are for true-positive predictions.

Figure 6:

Example of three-dimensional lesion identifications after automatic linking of adjacent two-dimensional boxes. Several consecutive held-out test images from one MRI series are shown through the posterior fossa, simulating model performance on a full stack of contiguous axial postcontrast T1-weighted images. Yellow boxes denote the predictions of the Mask region-based convolutional neural network model after self-labeling. Each prediction box includes an automatically generated lesion identification and confidence score. Green boxes denote ground truth annotations generated from mined tumor line annotations.