Development of a Deep Learning Algorithm for Periapical Disease Detection in Dental Radiographs

Abstract

Periapical radiolucencies, which can be detected on panoramic radiographs, are one of the most common radiographic findings in dentistry and have a differential diagnosis including infections, granuloma, cysts and tumors. In this study, we seek to investigate the ability with which 24 oral and maxillofacial (OMF) surgeons assess the presence of periapical lucencies on panoramic radiographs, and we compare these findings to the performance of a predictive deep learning algorithm that we have developed using a curated data set of 2902 de-identified panoramic radiographs. The mean diagnostic positive predictive value (PPV) of OMF surgeons based on their assessment of panoramic radiographic images was 0.69(± 0.13), indicating that dentists on average falsely diagnose 31% of cases as radiolucencies. However, the mean diagnostic true positive rate (TPR) was 0.51(± 0.14), indicating that on average 49% of all radiolucencies were missed. We demonstrate that the deep learning algorithm achieves a better performance than 14 of 24 OMF surgeons within the cohort, exhibiting an average precision of 0.60(± 0.04), and an F₁ score of 0.58(± 0.04) corresponding to a PPV of 0.67(± 0.05) and TPR of 0.51(± 0.05). The algorithm, trained on limited data and evaluated on clinically validated ground truth, has potential to assist OMF surgeons in detecting periapical lucencies on panoramic radiographs.

Keywords: artificial intelligence; computer-assisted; diagnosis; image interpretation; machine learning; panoramic radiograph; radiography.

Figure A1

Example 1 of a panoramic radiographic image (preprocessed for model input) selected from the test data set with overlays of the ground truth contours (Ground Truth), the intensity map output produced by our model (Model Output) and locations produced by our post-processing procedure (Postprocessed Output). Only predictions with a confidence score greater than 0.25 are displayed as an example (this threshold was selected to maximize the F₁ score on the validation data set).

Figure A2

Example 2; see Figure A1 for details.

Figure A3

Example 3; see Figure A1 for details.

Figure A4

Example 4; see Figure A1 for details.

Figure A5

Example of a preprocessed panoramic radiographic image, selected from the test data set, with overlays of the ground truth contours (solid) and four-pixel error tolerance regions (dashed).

Figure 1

Distribution of radiolucent periapical alterations per image for the training data set, the validation data set, and the testing data set.

Figure 2

Examples of panoramic radiographic images (preprocessed for model input) selected from the test data set with overlays of the ground truth contours (Ground Truth), the intensity map output produced by our model (Model Output) and locations produced by our post-processing procedure (Postprocessed Output). Only predictions with a confidence score greater than 0.25 are displayed (this threshold was selected to maximize the F₁ score on the validation data set). Higher resolution versions of these images are provided in Figure A1, Figure A2, Figure A3 and Figure A4.

Figure 3

Performance stratified by self-reported years of experience in diagnosing panoramic radiographs (lines indicate median, boxes span the first and third quartiles and fences span the total range). Groups contain 9 (≤4 years), 6 (4–8 years), and 9 (≥8 years) OMF surgeons, respectively.

Figure 4

Comparison of 24 OMF surgeon and model predictions in terms of F₁ score on the testing data set. The model threshold was chosen so that the F₁ score was maximized on the validation data set. Standard errors (whiskers and uncertainty bands) were computed via a jackknife analysis.

Figure 5

Comparison of 24 OMF surgeon and model performance on the test data set. Standard errors (whiskers), computed via a jackknife analysis. The curve of constant F₁ score equal to 0.58 shown is used to compare performance results in Figure 3.

Figure 6

Comparison of confidence score rankings for positive condition cases (left) and negative condition cases (right) produced by the model (axis labeled Deep Learning Model) and cohort of OMF surgeons (axis labeled Cohort of OMF surgeons). Regions of interest that are scored most (least) likely to be a radiolucent periapical alteration have highest (lowest) rank.