High-quality expert annotations enhance artificial intelligence model accuracy for osteosarcoma X-ray diagnosis

Abstract

Primary malignant bone tumors, such as osteosarcoma, significantly affect the pediatric and young adult populations, necessitating early diagnosis for effective treatment. This study developed a high-performance artificial intelligence (AI) model to detect osteosarcoma from X-ray images using highly accurate annotated data to improve diagnostic accuracy at initial consultations. Traditional models trained on unannotated data have shown limited success, with sensitivities of approximately 60%-70%. In contrast, our model used a data-centric approach with annotations from an experienced oncologist, achieving a sensitivity of 95.52%, specificity of 96.21%, and an area under the curve of 0.989. The model was trained using 468 X-ray images from 31 osteosarcoma cases and 378 normal knee images with a strategy to maximize diversity in the training and validation sets. It was evaluated using an independent dataset of 268 osteosarcoma and 554 normal knee images to ensure generalizability. By applying the U-net architecture and advanced image processing techniques such as renormalization and affine transformations, our AI model outperforms existing models, reducing missed diagnoses and enhancing patient outcomes by facilitating earlier treatment. This study highlights the importance of high-quality training data and advocates a shift towards data-centric AI development in medical imaging. These insights can be extended to other rare cancers and diseases, underscoring the potential of AI in transforming diagnostic processes in oncology. The integration of this AI model into clinical workflows could support physicians in early osteosarcoma detection, thereby improving diagnostic accuracy and patient care.

Keywords: artificial intelligence; clinical decision support; diagnostic imaging; image annotation; osteosarcoma detection.

FIGURE 1

Illustration of the network architecture used in our study, based on U‐Net shaped convolutional neural networks. The architecture processes a 2D input image through several steps, including pixel de‐shuffle before the first convolution and pixel shuffle after the last convolution to reduce the total number of network parameters. Convolutional operations are performed in each unit, referred to as a “Res block,” which is similar to a residual block with a skip connection. The network output employs multiresolution maps, akin to the U‐Net architecture.

FIGURE 2

Evaluation of artificial intelligence (AI)‐based lesion detection in X‐ray images. (A) Score distribution. Histogram showing the frequency distribution of lesion detection probabilities assigned by the AI model to the test set of X‐ray images, with an overlaid kernel density estimate indicating the score distribution pattern. (B) Receiver operating characteristic (ROC) curve. The ROC curve for the AI model, depicting the true positive rate (TPR) against the false positive rate (FPR) at various threshold settings. The area under the curve (AUC) is 0.989, illustrating the diagnostic accuracy of the model.

FIGURE 3

Osteosarcoma radiographs of the distal femur with artificial intelligence (AI) analysis results. (A) Original radiograph. X‐ray image of a patient's knee with osteosarcoma in the distal femur, where the tumor may be difficult to detect by untrained observers. (B) AI‐processed radiograph. X‐ray image with superimposed lesion detection; the red outline marks the tumor boundary as determined by expert human annotation, and the green outline shows the lesion area as identified by the AI model, highlighting the utility of the model in detecting subtle osteosarcoma manifestations. The lesion detection score of the AI model was 0.93, and the Dice coefficient was 0.63.

FIGURE 4

Scatter plot illustrating the correlation between lesion detection scores using artificial intelligence (AI) and Dice coefficients. The plot maps the lesion detection score assigned by the AI (x‐axis) against the Dice coefficient (y‐axis), which measures the agreement between the AI‐detected lesions and the expert‐annotated ground truth. A significant positive correlation (Pearson's correlation, 0.68; p < 0.001) shows that higher detection probabilities were generally associated with higher Dice coefficients, indicating reliable lesion identification using AI.

FIGURE 5

Artificial intelligence (AI) segmentation results with high and low Dice coefficients. For each case (A, B): left, original radiograph of the distal femur showing an osteosarcoma lesion; right, AI‐processed image with superimposed lesion detection. The red outline indicates the tumor boundary, as determined by expert human annotation, whereas the green outline represents the lesion area identified by the AI model. (A) High Dice coefficient case. The lesion detection score of the AI model was 0.99, and the Dice coefficient was 0.94, showing close agreement between AI segmentation and expert annotation. (B) Low Dice coefficient case. The AI model's lesion detection score was 0.82 and the Dice coefficient was 0.46. Despite the lower Dice coefficients, the AI model still identifies the primary area of concern, indicating its ability to detect subtle lesions even when full segmentation is challenging.

FIGURE 6

Osteosarcoma radiographs of the distal femur on whole lower extremity images and artificial intelligence (AI) reading results. (A) Original radiograph. A comprehensive full‐leg X‐ray depicting the distal femur, where a lesion is present but may be difficult to discern due to its small size in the context of the whole limb. (B) AI‐enhanced radiograph. X‐rays with AI detection overlays. The red contour represents the expert‐determined lesion boundary, while the green contour indicates the AI‐identified osteosarcoma lesion. The green markings in the pelvic area, which do not correspond to the red outline, indicate regions where the AI incorrectly identified intestinal shadows as potential osteosarcoma lesions.