Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Nov;309(2):e231693.
doi: 10.1148/radiol.231693.

Coarse Race and Ethnicity Labels Mask Granular Underdiagnosis Disparities in Deep Learning Models for Chest Radiograph Diagnosis

Preetham Bachina  1 , Sean P Garin  1 , Pranav Kulkarni  1 , Adway Kanhere  1 , Jeremias Sulam  1 , Vishwa S Parekh  1 , Paul H Yi  1
Affiliations

Coarse Race and Ethnicity Labels Mask Granular Underdiagnosis Disparities in Deep Learning Models for Chest Radiograph Diagnosis

Preetham Bachina et al. Radiology. 2023 Nov.

Abstract

See also the editorial by Nikolic in this issue.

PubMed Disclaimer

Conflict of interest statement

Disclosures of conflicts of interest: P.B. No relevant relationships. S.P.G. No relevant relationships. P.K. No relevant relationships. A.K. No relevant relationships. J.S. No relevant relationships. V.S.P. No relevant relationships. P.H.Y. Grants to institution from the National Institutes of Health/National Cancer Institute, RSNA, American College of Radiology, and University of Maryland Medical System Innovation Challenge; consulting fees from Bunkerhill Health and FH Ortho; payment for lectures from the Formosa Association for the Surgery of Trauma and Chang Gung Memorial Hospital and the University of Texas Southwestern; support for attending meetings from the Society of Imaging Informatics in Medicine (SIIM), Society of Nuclear Medicine and Molecular Imaging, and Sociedade Paulista de Radiologia; vice chair of the SIIM Annual Meeting Program Planning Committee and associate editor of Radiology: Artificial Intelligence; stock or stock options in Bunkerhill Health.

Figures

None
Graphical abstract
Forest plots show granular underdiagnosis rates (“no finding” label false-positive rate [FPR]) for models trained on MIMIC-CXR and CheXpert. Points show averages, and solid lines indicate 95% CIs for granular group false-positive rates in the results of the five models. Dashed lines and shaded regions show averages and 95% CIs, respectively, for coarse groups. Granular groups labeled with an asterisk are the patients who only reported a coarse race or ethnicity.
Forest plots show granular underdiagnosis rates (“no finding” label false-positive rate [FPR]) for models trained on MIMIC-CXR and CheXpert. Points show averages, and solid lines indicate 95% CIs for granular group false-positive rates in the results of the five models. Dashed lines and shaded regions show averages and 95% CIs, respectively, for coarse groups. Granular groups labeled with an asterisk are the patients who only reported a coarse race or ethnicity.
See this image and copyright information in PMC

Comment in

References

    1. Seyyed-Kalantari L , Zhang H , McDermott MBA , Chen IY , Ghassemi M . Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations . Nat Med 2021. ; 27 ( 12 ): 2176 – 2182 . - PMC - PubMed
    1. Movva R , Shanmugam D , Hou K , et al. . Coarse race data conceals disparities in clinical risk score performance . arXiv 2304.09270 [preprint] https://arxiv.org/abs/2304.09270. Posted April 18, 2023. Accessed June 2023.
    1. Jorde LB , Wooding SP . Genetic variation, classification and ‘race’ . Nat Genet 2004. ; 36 ( 11 Suppl ): S28 – S33 . - PubMed
    1. Johnson AEW , Pollard TJ , Berkowitz SJ , et al. . MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports . Sci Data 2019. ; 6 ( 1 ): 317 . - PMC - PubMed
    1. Irvin J , Rajpurkar P , Ko M , et al. . CheXpert: a large chest radiograph dataset with uncertainty labels and expert comparison . arXiv 1901.07031 [preprint] http://arxiv.org/abs/1901.07031. Posted January 21, 2019. Accessed February 7, 2022.

LinkOut - more resources