Deep learning for pediatric chest x-ray diagnosis: Repurposing a commercial tool developed for adults

Abstract

The number of commercially available artificial intelligence (AI) tools to support radiological workflows is constantly increasing, yet dedicated solutions for children are largely unavailable. Here, we repurposed an AI-tool developed for chest radiograph interpretation in adults (Lunit INSIGHT CXR) and investigated its diagnostic performance in a real-world pediatric clinical dataset. 958 consecutive frontal chest radiographs of children aged 2-14 years were included and analyzed with the commercially available AI-tool. The reference standard was determined in a dedicated reading session by a board-certified radiologist. The original reports validated by specialized pediatric radiologists, were considered as second readings. All discordant findings were reanalyzed in consensus. The diagnostic performance of the AI-tool was validated using standard measures of accuracy. For this, the continuous AI output (ranging from 0-100) was binarized using vendor recommended thresholds recommended for adults and optimized thresholds identified for children. Relevant findings were defined as consolidation, atelectasis, nodule, cardiomegaly, mediastinal widening due to mass, pleural effusion and pneumothorax. 200 radiographs [20.9%] demonstrated at least one relevant pathology. Using the adult threshold, the AI-tool showed a high performance for all relevant findings with an AUC 0.94 (95% CI: 0.92-0.95) and. In stratified analysis by age (2-7 vs. 7-14-years-old) a significantly higher performance (p < 0.001) was found for older children with an AUC of 0.96 (95% CI: 0.94-0.98) with a sensitivity and specificity of 87.5% and 82.3% respectively, which further increased using optimized thresholds for children. Repurposing existing AI-tools developed for adult application to pediatric patients could support clinical workflows until dedicated solutions become available.

Fig 1. Brief summary highlighting the study methology. a) AI Tool Development and Pediatric Repurposing: The AI tool was originally trained and validated using a large dataset of adult chest radiographs.

For pediatric validation, the tool was retrospectively tested on 958 pediatric chest radiographs (CXR) from children aged 2–14 years. b) Diagnostic Performance Analysis: The AI tool’s diagnostic performance in children was assessed using vendor-recommended thresholds, stratified by age groups (2–6 and 7–14 years), and optimized pediatric-specific thresholds.

Fig 2. Example of reference standard and AI output.

A) Reference standard with annotated finding by board-certified radiologist specializing in thoracic imaging. Blue marker indicates a consolidation in the right lower zone. The box in the upper right corner shows the annotation tool of the image-processing platform NORA. B) AI output with grayscale map showing consolidation (Csn) with an abnormality score of 68% in the right lower zone, considered as a true positive finding.

Fig 3. Image examples representing strengths and limitations of the AI tool.

A) AI output in a 10-year-old patient with a history of Ewing sarcoma of the 1st right rib. The area was highlighted as pathologic with an abnormality score of 73% and classified as consolidation (Csn), fibrosis (Fib) and nodule (Ndl), which most closely resemble the findings the AI tool was developed for. B) Image of a 4-year-old child with a venolymphatic malformation of the chest wall. Similar to A an abnormality was correctly detected but erroneously classified as effusion (PEf) and consolidation (Csn) as it was beyond the application of the AI tool.

Fig 4. Definition of optimal thresholds for the performance of the AI-tool in children.

The dotted blue line represents the pre-defined vendor recommended threshold of 15, which is based on the optimal threshold identified for adults to dichotomize the continuous AI-output (0-100). The green diamonds show optimized cut-offs calculated for maximizing the sum of sensitivity and specificity.. The performance metrics based on adult threshold of 15 (blue) and optimized cutoffs for children (green) are shown in the column on the right side (sens = sensitivity, spec = specificity, PPV = positive predictive value, NPV = negative predictive value).