Extracting decision-making features from the unstructured eye movements of clinicians on glaucoma OCT reports and developing AI models to classify expertise

Abstract

This study aimed to investigate the eye movement patterns of ophthalmologists with varying expertise levels during the assessment of optical coherence tomography (OCT) reports for glaucoma detection. Objectives included evaluating eye gaze metrics and patterns as a function of ophthalmic education, deriving novel features from eye-tracking, and developing binary classification models for disease detection and expertise differentiation. Thirteen ophthalmology residents, fellows, and clinicians specializing in glaucoma participated in the study. Junior residents had less than 1 year of experience, while senior residents had 2-3 years of experience. The expert group consisted of fellows and faculty with over 3 to 30+ years of experience. Each participant was presented with a set of 20 Topcon OCT reports (10 healthy and 10 glaucomatous) and was asked to determine the presence or absence of glaucoma and rate their confidence of diagnosis. The eye movements of each participant were recorded as they diagnosed the reports using a Pupil Labs Core eye tracker. Expert ophthalmologists exhibited more refined and focused eye fixations, particularly on specific regions of the OCT reports, such as the retinal nerve fiber layer (RNFL) probability map and circumpapillary RNFL b-scan. The binary classification models developed using the derived features demonstrated high accuracy up to 94.0% in differentiating between expert and novice clinicians. The derived features and trained binary classification models hold promise for improving the accuracy of glaucoma detection and distinguishing between expert and novice ophthalmologists. These findings have implications for enhancing ophthalmic education and for the development of effective diagnostic tools.

Keywords: eye-tracking; fixations; glaucoma; neural networks; optical coherence tomography; unsupervised clustering.

Figure 1

Left: Input structure for sequence graph transform (SGT) embedding that associates each fixation ID with a region of interest (ROI) from the OCT report. For example, the clinicians fixated on Region 1 (circumpapillary retinal nerve fiber layer (RNFL) b-scan) for IDs 1, 2, 3, 30, 31. Right: OCT report with the seven labeled regions of interest (ROIs), highlighted in yellow for enhanced contrast and visibility: Region 1 corresponds to the circumpapillary RNFL b-scan, Region 2 corresponds to the en-face slab, Region 3 corresponds to the RNFL thickness map, Region 4 corresponds to the sectoral thickness pie charts, Region 5 corresponds to the RNFL probability map, Region 6 corresponds to the retinal ganglion cell layer (GCL) + thickness map, Region 7 corresponds to the GCL + probability map.

Figure 2

Histogram of fixation counts on OCT full reports read by participants.

Figure 3

Mann Whitney U test values of p for total fixation count comparisons across healthy and glaucomatous OCT reports by regions of interest (ROIs) within expertise groups (1: jr. resident, 2: sr. resident, 3: expert). Expert group exhibited significant differences in the number of fixations for Regions 1 and 5 (values of p indicated in red bold font) which correspond to the circumpapillary retinal nerve fiber layer (RNFL) b-scan and the RNFL probability map, respectively.

Figure 4

Model performance for binary classification based on region of interest (ROI) fixation positional encoding and based on sequence graph transform (SGT) embedding approach (average test accuracy and AUC based on 50 randomized train-test splits).

Figure 5

(A) Results for principal components analysis (PCA) application on embeddings and (B) visualizing the clusters of regions of interest (ROIs) in two dimensions.

Figure 6

Visual representation of the ROIs contained in each cluster mapped onto an OCT full report. Red cluster contains Regions 2, 3, 4, 6, and 7 while Regions 1 (blue) and 5 (green) were clustered separately.

Figure 7

Frequency of same cluster groupings across training expertise groups for (A) healthy optical coherence tomography (OCT) reports and (B) glaucomatous OCT reports. The rows highlighted in yellow indicate the top 1 or 2 groups with highest common cluster frequencies observed in healthy and glaucomatous reports, respectively. Note that the highest cluster agreement is between sr. residents and experts.

Figure 8

Region permutation importance chart using model weights in test data for (A) Glaucoma vs. Healthy model and (B) Expert vs. Novice Model. Both models illustrate Region 5 (RNFL probability map) as the most important feature.