Machine learning derived retinal pigment score from ophthalmic imaging shows ethnicity is not biology

Abstract

Few metrics exist to describe phenotypic diversity within ophthalmic imaging datasets, with researchers often using ethnicity as a surrogate marker for biological variability. We derived a continuous, measured metric, the retinal pigment score (RPS), that quantifies the degree of pigmentation from a colour fundus photograph of the eye. RPS was validated using two large epidemiological studies with demographic and genetic data (UK Biobank and EPIC-Norfolk Study) and reproduced in a Tanzanian, an Australian, and a Chinese dataset. A genome-wide association study (GWAS) of RPS from UK Biobank identified 20 loci with known associations with skin, iris and hair pigmentation, of which eight were replicated in the EPIC-Norfolk cohort. There was a strong association between RPS and ethnicity, however, there was substantial overlap between each ethnicity and the respective distributions of RPS scores. RPS decouples traditional demographic variables from clinical imaging characteristics. RPS may serve as a useful metric to quantify the diversity of the training, validation, and testing datasets used in the development of AI algorithms to ensure adequate inclusion and explainability of the model performance, critical in evaluating all currently deployed AI models. The code to derive RPS is publicly available at: https://github.com/uw-biomedical-ml/retinal-pigmentation-score .

Fig. 1. Schematic showing the method to generate the retinal pigmentation score (RPS) from a colour fundus image.

Input images are fed into the deep learning algorithm to generate segmentation masks. These are added together to make a retinal background mask, which is then transformed into L,a,b colorspace. The chromaticity vectors are then extracted and transformed by the principal component analysis model to create the RPS. Created with Biorender.com.

Fig. 2. Representative fundus photos with associated RPS.

a Randomly sampled colour fundus photographs from each UK Biobank self-reported ethnicity and from the Tanzanian, Australian, and Chinese (ODIR) datasets, sorted by quintiles of retinal pigment score (RPS) across the entire distribution of RPS for the UK Biobank cohort. The RGB colour of the pixel value that is converted into RPS as well as the RPS is shown at the bottom of each fundus photograph. Black spaces represent when there are no suitable images within the respective ethnicity subgroup and quintile b Normalised kernel density estimation plot of the distribution of RPS for all participants grouped by self-reported ethnicity as reported in the UK Biobank as well as the Tanzanian, Australian, and Chinese (ODIR) datasets. Relative frequencies are normalised so the area under each curve is equal for each ethnicity subgroup. The subpanel consists of examples where for a given RPS and the a,b values in the CIELAB colour space are constant but the L vector changes. The x-axis is shared in both subpanels. Source data are provided as a Source Data file.

Fig. 3. Manhattan plot of GWAS results from the discovery cohort (UKBiobank, n = 37067).

The Y-axis represents the two-sided p-values from the linear mixed effects model. Lead variants identified by GCTA-COJO are annotated with the nearest gene. Points are truncated at −log10(p) = 70 for clarity. The dashed red line indicates genome-wide significance (p = 5 × 10⁻⁸) which is adjusted for multiple comparisons and the p-values are two-sided and calculated with the z-statistic. Source data are provided as a Source Data file.

Fig. 4. Comparison of betas for lead variants identified from the discovery and replication cohort.

Comparison of betas expressed as change in standard deviation of mean RPS for lead variants identified from the discovery (UK Biobank, n = 37067) genome-wide association study (GWAS) with their corresponding betas in the replication (EPIC-Norfolk, n = 4273) analysis, with 95% confidence intervals. Betas in the cohort were calculated using a generalised linear mixed model, adjusting for age, sex and the first ten principal components. P-values are two-sided, calculated from the z-statistic and corrected for multiple comparisons. Variants meeting the Bonferroni-adjusted replication significance threshold (p = 0.05/ variants) in the EPIC-Norfolk GWAS are shaded black. The nearest gene is annotated for variants achieving genome-wide significance. Source data are provided as a Source Data file.