Deep Learning Model to Classify and Monitor Idiopathic Scoliosis in Adolescents Using a Single Smartphone Photograph

Abstract

Importance: Adolescent idiopathic scoliosis (AIS) is the most common pediatric spinal disorder. Routine physical examinations by trained personnel are critical to diagnose severity and monitor curve progression in AIS. In the presence of concerning malformation, radiographs are necessary for diagnosis or follow-up, guiding further management, such as bracing correction for moderate malformation and spine surgery for severe malformation. If left unattended, progressive deterioration occurs in two-thirds of patients, leading to significant health concerns for growing children.

Objective: To assess the ability of an open platform application (app) using a validated deep learning model to classify AIS severity and curve type, as well as identify progression.

Design, setting, and participants: This diagnostic study was performed with data from radiographs and smartphone photographs of the backs of adolescent patients at spine clinics. The ScolioNets deep learning model was developed and validated in a prospective training cohort, then incorporated and tested in the AlignProCARE open platform app in 2022. Ground truths (GTs) included severity, curve type, and progression as manually annotated by 2 experienced spine specialists based on the radiographic examinations of the participants' spines. The GTs and app results were blindly compared with another 2 spine surgeons' assessments of unclothed back appearance. Data were analyzed from October 2022 to February 2023.

Exposure: Acquisitions of unclothed back photographs using a mobile app.

Main outcomes and measures: Outcomes of interest were classification of AIS severity and progression. Quantitative statistical analyses were performed to assess the performance of the deep learning model in classifying the deformity as well as in distinguishing progression during 6-month follow-up.

Results: The training data set consisted of 1780 patients (1295 [72.8%] female; mean [SD] age, 14.3 [3.3] years), and the prospective testing data sets consisted of 378 patients (279 [73.8%] female; mean [SD] age, 14.3 [3.8] years) and 376 follow-ups (294 [78.2%] female; mean [SD] age, 15.6 [2.9] years). The model recommended follow-up with an area under receiver operating characteristic curve (AUC) of 0.839 (95% CI, 0.789-0.882) and considering surgery with an AUC of 0.902 (95% CI, 0.859-0.936), while showing good ability to distinguish among thoracic (AUC, 0.777 [95% CI, 0.745-0.808]), thoracolumbar or lumbar (AUC, 0.760 [95% CI, 0.727-0.791]), or mixed (AUC, 0.860 [95% CI, 0.834-0.887]) curve types. For follow-ups, the model distinguished participants with or without curve progression with an AUC of 0.757 (95% CI, 0.630-0.858). Compared with both surgeons, the model could recognize severities and curve types with a higher sensitivity (eg, sensitivity for recommending follow-up: model, 84.88% [95% CI, 75.54%-91.70%]; senior surgeon, 44.19%; junior surgeon, 62.79%) and negative predictive values (NPVs; eg, NPV for recommending follow-up: model, 89.22% [95% CI, 84.25%-93.70%]; senior surgeon, 71.76%; junior surgeon, 79.35%). For distinguishing curve progression, the sensitivity and NPV were comparable with the senior surgeons (sensitivity, 63.33% [95% CI, 43.86%-80.87%] vs 77.42%; NPV, 68.57% [95% CI, 56.78%-78.37%] vs 72.00%). The junior surgeon reported an inability to identify curve types and progression by observing the unclothed back alone.

Conclusions: This diagnostic study of adolescent patients screened for AIS found that the deep learning app had the potential for out-of-hospital accessible and radiation-free management of children with scoliosis, with comparable performance as spine surgeons experienced in AIS management.

Figure 1.. Participant Recruitment Flowchart

Figure 2.. ROC Curve and Confusion Matrix for Prospective Test

B, The confusion matrix corresponds to the final follow-up severity classification results. C, The confusion matrix corresponds to the final consider surgery severity classification results. AUC indicates area under the receiver operating characteristic (ROC) curve; T, thoracic; TL/L, thoracolumbar and lumbar.