Automated landmark identification for diagnosis of the deformity using a cascade convolutional neural network (FlatNet) on weight-bearing lateral radiographs of the foot

Abstract

Landmark detection in flatfoot radiographs is crucial in analyzing foot deformity. Here, we evaluated the accuracy and efficiency of the automated identification of flatfoot landmarks using a newly developed cascade convolutional neural network (CNN) algorithm, Flatfoot Landmarks AnnoTating Network (FlatNet). A total of 1200 consecutive weight-bearing lateral radiographs of the foot were acquired. The first 1050 radiographs were used as the training and tuning, and the following 150 radiographs were used as the test sets, respectively. An expert orthopedic surgeon (A) manually labeled ground truths for twenty-five anatomical landmarks. Two orthopedic surgeons (A and B, each with eight years of clinical experience) and a general physician (GP) independently identified the landmarks of the test sets using the same method. After two weeks, observers B and GP independently identified the landmarks once again using the developed deep learning CNN model (DLm). The X- and Y-coordinates and the mean absolute distance were evaluated. The average differences (mm) from the ground truth were 0.60 ± 0.57, 1.37 ± 1.28, and 1.05 ± 1.23 for the X-coordinate, and 0.46 ± 0.59, 0.97 ± 0.98, and 0.73 ± 0.90 for the Y-coordinate in DLm, B, and GP, respectively. The average differences (mm) from the ground truth were 0.84 ± 0.73, 1.90 ± 1.34, and 1.42 ± 1.40 for the absolute distance in DLm, B, and GP, respectively. Under the guidance of the DLm, the overall differences (mm) from the ground truth were enhanced to 0.87 ± 1.21, 0.69 ± 0.74, and 1.24 ± 1.31 for the X-coordinate, Y-coordinate, and absolute distance, respectively, for observer B. The differences were also enhanced to 0.74 ± 0.73, 0.57 ± 0.63, and 1.04 ± 0.85 for observer GP. The newly developed FlatNet exhibited better accuracy and reliability than the observers. Furthermore, under the FlatNet guidance, the accuracy and reliability of the human observers generally improved.

Keywords: Cascade convolutional neural network; Deep learning; Diagnosis; Flatfoot; Landmark detection.