Ethnicity is not biology: retinal pigment score to evaluate biological variability from ophthalmic imaging using machine learning

Abstract

Background: Few metrics exist to describe phenotypic diversity within ophthalmic imaging datasets, with researchers often using ethnicity as an inappropriate marker for biological variability.

Methods: We derived a continuous, measured metric, the retinal pigment score (RPS), that quantifies the degree of pigmentation from a colour fundus photograph of the eye. RPS was validated using two large epidemiological studies with demographic and genetic data (UK Biobank and EPIC-Norfolk Study).

Findings: A genome-wide association study (GWAS) of RPS from UK Biobank identified 20 loci with known associations with skin, iris and hair pigmentation, of which 8 were replicated in the EPIC-Norfolk cohort. There was a strong association between RPS and ethnicity, however, there was substantial overlap between each ethnicity and the respective distributions of RPS scores.

Interpretation: RPS serves to decouple traditional demographic variables, such as ethnicity, from clinical imaging characteristics. RPS may serve as a useful metric to quantify the diversity of the training, validation, and testing datasets used in the development of AI algorithms to ensure adequate inclusion and explainability of the model performance, critical in evaluating all currently deployed AI models. The code to derive RPS is publicly available at: https://github.com/uw-biomedical-ml/retinal-pigmentation-score.

Funding: The authors did not receive support from any organisation for the submitted work.

Figure 1.

Schematic showing the method to generate the retinal pigmentation score (RPS) from a colour fundus image. Input images are fed into the deep learning algorithm to generate segmentation masks. These are added together to make a retinal background mask, which is then transformed into L,a,b colorspace. The chromaticity vectors are then extracted and transformed by a principal component analysis model to create the RPS.

Figure 2.

a. Randomly sampled colour fundus photographs from each self-reported ethnicities by quintiles of retinal pigment score (RPS) across the entire distribution of RPS for the UK Biobank cohort and each associated self-reported ethnicity of the participant. The retinal background colour and the RPS is shown at the bottom of each fundus photograph. b. Normalised kernel density estimation plot of the distribution of RPS for all participants grouped by self-reported ethnicity as reported in the UK Biobank. Relative frequencies are normalised so the area under each curve is equal for each ethnicity.

Figure 3.

Manhattan plot of GWAS results. Lead variants identified by GCTA-COJO are annotated with the nearest gene. Points are truncated at −log10(p) = 70 for clarity. The dashed red line indicates genome-wide significance (p = 5 x 10⁻⁸).

Figure 4.

Comparison of betas expressed as change in standard deviation of mean RPS for lead variants identified from the discovery (UK Biobank) genome-wide association study (GWAS) with their corresponding betas in the replication (EPIC-Norfolk) analysis, with 95% confidence intervals. Variants meeting the Bonferroni-adjusted replication significance threshold (p = 0·05/17 variants) in the EPIC-Norfolk GWAS are shaded black. The nearest gene is annotated for variants achieving genome-wide significance.