Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Jun 1;141(6):543-552.
doi: 10.1001/jamaophthalmol.2023.1310.

Association of Biomarker-Based Artificial Intelligence With Risk of Racial Bias in Retinal Images

Aaron S Coyner  1 , Praveer Singh  2   3 , James M Brown  4 , Susan Ostmo  1 , R V Paul Chan  5 , Michael F Chiang  6 , Jayashree Kalpathy-Cramer  2   3 , J Peter Campbell  1 ; Imaging and Informatics in Retinopathy of Prematurity Consortium
Collaborators, Affiliations

Association of Biomarker-Based Artificial Intelligence With Risk of Racial Bias in Retinal Images

Aaron S Coyner et al. JAMA Ophthalmol. .

Abstract

Importance: Although race is a social construct, it is associated with variations in skin and retinal pigmentation. Image-based medical artificial intelligence (AI) algorithms that use images of these organs have the potential to learn features associated with self-reported race (SRR), which increases the risk of racially biased performance in diagnostic tasks; understanding whether this information can be removed, without affecting the performance of AI algorithms, is critical in reducing the risk of racial bias in medical AI.

Objective: To evaluate whether converting color fundus photographs to retinal vessel maps (RVMs) of infants screened for retinopathy of prematurity (ROP) removes the risk for racial bias.

Design, setting, and participants: The retinal fundus images (RFIs) of neonates with parent-reported Black or White race were collected for this study. A u-net, a convolutional neural network (CNN) that provides precise segmentation for biomedical images, was used to segment the major arteries and veins in RFIs into grayscale RVMs, which were subsequently thresholded, binarized, and/or skeletonized. CNNs were trained with patients' SRR labels on color RFIs, raw RVMs, and thresholded, binarized, or skeletonized RVMs. Study data were analyzed from July 1 to September 28, 2021.

Main outcomes and measures: Area under the precision-recall curve (AUC-PR) and area under the receiver operating characteristic curve (AUROC) at both the image and eye level for classification of SRR.

Results: A total of 4095 RFIs were collected from 245 neonates with parent-reported Black (94 [38.4%]; mean [SD] age, 27.2 [2.3] weeks; 55 majority sex [58.5%]) or White (151 [61.6%]; mean [SD] age, 27.6 [2.3] weeks, 80 majority sex [53.0%]) race. CNNs inferred SRR from RFIs nearly perfectly (image-level AUC-PR, 0.999; 95% CI, 0.999-1.000; infant-level AUC-PR, 1.000; 95% CI, 0.999-1.000). Raw RVMs were nearly as informative as color RFIs (image-level AUC-PR, 0.938; 95% CI, 0.926-0.950; infant-level AUC-PR, 0.995; 95% CI, 0.992-0.998). Ultimately, CNNs were able to learn whether RFIs or RVMs were from Black or White infants regardless of whether images contained color, vessel segmentation brightness differences were nullified, or vessel segmentation widths were uniform.

Conclusions and relevance: Results of this diagnostic study suggest that it can be very challenging to remove information relevant to SRR from fundus photographs. As a result, AI algorithms trained on fundus photographs have the potential for biased performance in practice, even if based on biomarkers rather than raw images. Regardless of the methodology used for training AI, evaluating performance in relevant subpopulations is critical.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest Disclosures: Dr Coyner reported receiving personal fees from Boston AI outside the submitted work. Dr Chan reported being an owner of Siloam Vision, receiving personal fees from Alcon and Genentech, and being on the scientific advisory board for Phoenix Technology Group outside the submitted work. Drs Chan, Chiang, Kalpathy-Cramer, and Campbell reported receiving research support from Genentech. Drs Chan, Campbell, Coyner, and Kalpathy-Cramer reported potential receipt of royalties from the i-ROP DL system has been licensed to Boston AI Lab (Boston, Massachusetts) by Oregon Health & Science University, Massachusetts General Hospital, Northeastern University, and the University of Illinois, Chicago. Drs Chan and Campbell reported being equity owners of Siloam Vision. Dr Chiang reported being an employee of the National Institutes of Health (after 11/2020); receiving grant support from the National Institutes of Health (terminated 11/2020), the National Science Foundation (terminated 11/2020), and Genentech (terminated 11/2020); consultant fees from Novartis (terminated 11/2020); and being an equity owner of InTeleretina LLC (terminated 11/2020) outside the submitted work. Dr Kalpathy-Cramer reported receiving grants from the National Institutes of Health, GE Heathcare, and Genentech and consultant fees from Siloam Vision Inc outside the submitted work. Dr Campbell reported receiving grants from the National Institutes of Health, Genentech, RPB; being an equity owner of Siloam Vision; and receiving consultant fees from Boston AI (terminated 2021) outside the submitted work. No other disclosures were reported.

Figures

Figure 1.
Figure 1.. Retinal Fundus Images (RFIs) and Associated Retinal Vessel Maps (RVMs) of Black and White Infants
RFIs were collected using visible-light retinal photography from premature Black (A) and White (B) infants. From RFIs, RVMs were segmented with a u-net to, theoretically, nullify pigment- and luminance-associated information in Black (C) and White (D) infants.
Figure 2.
Figure 2.. Retinal Vessel Maps (RVMs) With Pixel Intensity Values (PIVs)
In addition to previous thresholding, PIVs greater than 10 or PIVs less than 75 and greater than 150 were zeroed. Binarized RVMs revealed that much information was still contained within RVMs containing only PIVs less than 10.
Figure 3.
Figure 3.. Histograms of the Mean Number of Segmented Pixels in Retinal Vessel Maps (RVMs) of Black and White Infants
The pixel intensity values (PIVs) segmented in raw RVMs in the thresholded (A) and skeletonized (B) maps with PIVs of 0 or greater are distinctly different between the eyes of Black and White infants. Increased thresholding of these RVMs in the thresholded (C) and skeletonized (D) maps with PIVs of 50 or greater results in far better overlap, but there are still differences. RVM thresholding is further increased to a PIV of 200 or greater in the thresholded (E) and skeletonized (F) maps. Removing the segmented vessel width via skeletonizing still resulted in differences between groups.
See this image and copyright information in PMC

Comment in

References

    1. Hall WJ, Chapman MV, Lee KM, et al. . Implicit racial/ethnic bias among health care professionals and its influence on health care outcomes: a systematic review. Am J Public Health. 2015;105(12):e60-e76. doi:10.2105/AJPH.2015.302903 - DOI - PMC - PubMed
    1. Chen J, Khazanchi R, Bearman G, Marcelin JR. Racial/ethnic inequities in health care–associated infections under the shadow of structural racism: narrative review and call to action. Curr Infect Dis Rep. 2021;23(10):17. doi:10.1007/s11908-021-00758-x - DOI - PMC - PubMed
    1. Hilbert A, Zenger M, Luck-Sikorski C, Brähler E. Weight stigma and disease and disability concepts of obesity: a survey of the German population. Obes Facts. 2021;14(5):463-470. doi:10.1159/000516494 - DOI - PMC - PubMed
    1. Bradford PS, Akyeampong D, Fleming MA II, Dacus AR, Chhabra AB, DeGeorge BR Jr. Racial and gender discrimination in hand surgery letters of recommendation. J Hand Surg Am. 2021;46(11):998-1005.e2. doi:10.1016/j.jhsa.2021.07.009 - DOI - PubMed
    1. Elk R. The intersection of racism, discrimination, bias, and homophobia toward African American sexual minority patients with cancer within the health care system. Cancer. 2021;127(19):3500-3504. doi:10.1002/cncr.33627 - DOI - PubMed

Publication types

LinkOut - more resources