Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Oct;113(10):3528-3534.
doi: 10.1111/cas.15511. Epub 2022 Aug 3.

Establishment of a deep-learning system to diagnose BI-RADS4a or higher using breast ultrasound for clinical application

Tetsu Hayashida  1 , Erina Odani  1 , Masayuki Kikuchi  1 , Aiko Nagayama  1 , Tomoko Seki  1 , Maiko Takahashi  1 , Noriyuki Futatsugi  2 , Akiko Matsumoto  3 , Takeshi Murata  4 , Rurina Watanuki  5 , Takamichi Yokoe  5 , Ayako Nakashoji  6 , Hinako Maeda  7 , Tatsuya Onishi  5 , Sota Asaga  8 , Takashi Hojo  9 , Hiromitsu Jinno  3 , Keiichi Sotome  7 , Akira Matsui  6 , Akihiko Suto  4 , Shigeru Imoto  8 , Yuko Kitagawa  1
Affiliations

Establishment of a deep-learning system to diagnose BI-RADS4a or higher using breast ultrasound for clinical application

Tetsu Hayashida et al. Cancer Sci. 2022 Oct.

Abstract

Although the categorization of ultrasound using the Breast Imaging Reporting and Data System (BI-RADS) has become widespread worldwide, the problem of inter-observer variability remains. To maintain uniformity in diagnostic accuracy, we have developed a system in which artificial intelligence (AI) can distinguish whether a static image obtained using a breast ultrasound represents BI-RADS3 or lower or BI-RADS4a or higher to determine the medical management that should be performed on a patient whose breast ultrasound shows abnormalities. To establish and validate the AI system, a training dataset consisting of 4028 images containing 5014 lesions and a test dataset consisting of 3166 images containing 3656 lesions were collected and annotated. We selected a setting that maximized the area under the curve (AUC) and minimized the difference in sensitivity and specificity by adjusting the internal parameters of the AI system, achieving an AUC, sensitivity, and specificity of 0.95, 91.2%, and 90.7%, respectively. Furthermore, based on 30 images extracted from the test data, the diagnostic accuracy of 20 clinicians and the AI system was compared, and the AI system was found to be significantly superior to the clinicians (McNemar test, p < 0.001). Although deep-learning methods to categorize benign and malignant tumors using breast ultrasound have been extensively reported, our work represents the first attempt to establish an AI system to classify BI-RADS3 or lower and BI-RADS4a or higher successfully, providing important implications for clinical actions. These results suggest that the AI diagnostic system is sufficient to proceed to the next stage of clinical application.

Keywords: AI diagnosis; BI-RADS; artificial intelligence; breast ultrasound; deep learning.

PubMed Disclaimer

Figures

FIGURE 1
FIGURE 1
Receiver‐operating characteristic (ROC) curve by possible thresholds of the confidence score for the detection in each image of Breast Imaging Reporting and Data System (BI‐RADS) 4a or higher. A, ROC curve with an area under the curve (AUC) of 0.95. B, Sensitivity and specificity with variations in thresholds of the confidence score
FIGURE 2
FIGURE 2
Sensitivity and specificity of diagnosis by artificial Intelligence (AI) and 20 clinicians for 30 images. X: diagnosis by each clinician, ▲: AI diagnosis, ●: average of clinicians
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Shahan CL, Layne GP. Advances in breast imaging with current screening recommendations and controversies. Obstet Gynecol Clin North Am. 2022;49:1‐33. - PubMed
    1. Berg WA, Blume JD, Cormack JB, et al. Combined screening with ultrasound and mammography vs mammography alone in women at elevated risk of breast cancer. JAMA. 2008;299:2151‐2163. - PMC - PubMed
    1. Lazarus E, Mainiero MB, Schepps B, Koelliker SL, Livingston LS. BI‐RADS lexicon for US and mammography: interobserver variability and positive predictive value. Radiology. 2006;239:385‐391. - PubMed
    1. Spak DA, Plaxco JS, Santiago L, Dryden MJ, Dogan BE. BI‐RADS([R]) fifth edition: a summary of changes. Diagn Interv Imaging. 2017;98:179‐190. - PubMed
    1. Abdullah N, Mesurolle B, El‐Khoury M, Kao E. Breast imaging reporting and data system lexicon for US: interobserver agreement for assessment of breast masses. Radiology. 2009;252:665‐672. - PubMed