Machine Learning-Based Automated Detection of Hydroxychloroquine Toxicity and Prediction of Future Toxicity Using Higher-Order OCT Biomarkers

Abstract

Objective: Despite guidelines for hydroxychloroquine (HCQ) toxicity screening, there are clear challenges to accurate detection and interpretation. In the current report, the feasibility of automated machine learning (ML)-based detection of HCQ retinopathy and prediction of progression to toxicity in eyes without preexisting toxicity has been described.

Design: Retrospective, longitudinal cohort study.

Subjects: Subjects on HCQ therapy.

Methods: This was an institutional review board-approved, retrospective, longitudinal image analysis of 388 subjects on HCQ. Multilayer, compartmental, retinal segmentation with ellipsoid zone (EZ) mapping was used to harvest quantitative spectral-domain (SD)-OCT biomarkers. Using a combination of clinical features (i.e., cumulative HCQ dose and the duration of therapy) and quantitative imaging biomarkers (e.g., volumetric EZ integrity and compartmental measurements), ML models were created to detect toxicity and predict progression based on ground-truth OCT-based toxicity readings by 2 masked retina specialists. Furthermore, 10-fold cross-validation was performed.

Main outcome measures: The model performance was visualized using receiver operator curves and calculating the area under the curve (AUC). The corresponding sensitivity and specificity values were evaluated for the feasibility of HCQ toxicity screening and prediction.

Results: The prevalence of HCQ toxicity in this cohort of 388 patients was 9.8% (n = 38). Twenty-one eyes progressed to toxicity during follow-up. OCT-based features (i.e., partial EZ attenuation, EZ volume, outer nuclear layer volume, and compartmental thicknesses) and clinical features (i.e., HCQ daily dose, HCQ cumulative dose, and duration of therapy) showed significant differences between the toxic and nontoxic groups. Percentage area with partial EZ attenuation (i.e., percentage of the macula with an EZ-retinal pigment epithelium thickness of ≤ 20 μm) was the most discriminating single feature (toxic, 35.7 ± 46.5%; nontoxic, 1.8 ± 4.4%; P < 0.0001). Using a random forest model, high-performance, automated toxicity detection was achieved, with a mean AUC of 0.97, sensitivity of 95%, and specificity of 91%. Furthermore, the toxicity progression prediction model had a mean AUC of 0.89, with a sensitivity and specificity of 90% and 80%, respectively.

Conclusions: This report described the feasibility of high-performance automated classification models that used a combination of clinical and quantitative SD-OCT biomarkers to detect HCQ retinal toxicity and predict progression to toxicity in cases without toxicity. Future work is needed to validate these findings in an independent dataset.

Keywords: Automated detection; Ellipsoid zone integrity; Hydroxychloroquine toxicity; OCT; Quantitative biomarkers for hydroxychloroquine toxicity.

Figure 1:

Multi-layer segmentation of retinal layers on OCT in a patient (A-C) without toxicity and (D-F) with toxicity; A) and D) OCT B scan corresponding to the selected slice from the volume with layer segmentation of the ILM (yellow line), ONL (green line), EZ (red line) and RPE (orange line) along with ONL-RPE thickness measurements (white brace) and EZ-RPE thickness measurements (red brace); B) and E) EZ-RPE thickness maps with the red line indicating selected B scan from the OCT volume (white arrows indicate loss of EZ integrity due to HCQ toxicity); C) and F) 3-dimensional reconstruction of the EZ-RPE layer visualizing changes in EZ integrity. OCT: Optical coherence tomography, ILM: Inner limiting membrane, ONL: Outer nuclear layer, EZ: Ellipsoid zone, RPE: Retinal pigment epithelium

Figure 2:

(A) Box-plot maps for the univariate analysis for clinical and imaging biomarkers comparing the toxic and non-toxic groups; (B) Receiver Operator Curve (ROC) for the toxicity detection model B with the mean area under curve (AUC) of 0.97 after 10-fold cross validation. HCQ: Hydroxychloroquine, EZ: Ellipsoid zone, VA: Visual Acuity, ONL: Outer nuclear layer, Point thickness: Layer thickness at a single point at 1 mm nasal or temporal to fovea

Figure 3:

(A) Box-plot maps for the univariate analysis for clinical and imaging biomarkers comparing the progressor and non-progressor groups (n=377); (B) Receiver Operator Curve (ROC) for the toxicity prediction model with the mean area under curve (AUC) of 0.87 after 10-fold cross validation. HCQ: Hydroxychloroquine, EZ: Ellipsoid zone, VA: Visual Acuity, ONL: Outer nuclear layer, Point thickness: Layer thickness at a single point at 1 mm nasal or temporal to fovea

Figure 4:

Toxicity detection model false negative (A-C) and false positive (D-G) EZ-RPE thickness maps with the red line indicating selected B scan from the OCT volume; (A,B) EZ-RPE thickness maps demonstrates classic perifoveal EZ loss consistent with hydroxychloroquine toxicity at both timepoints for this subject confirming the false negative classification. (C) B-scan shows corresponding structural findings with EZ loss. (D-G) EZ-RPE demonstrate some EZ inetegrity loss that at the second timepoint (E) may be more consistent with hydroxychloroquine toxicity; however neither scan was graded as toxic by masked review. (F, G) B-scans demonstrate abnormal retinal contour with staphylomatous alteration that may have impacted algorithm. Nevertheless, EZ attenuation does appear to be present on secondary review and may represent true toxicity (rather than a false positive). OCT: Optical coherence tomography, ILM: Inner limiting membrane, ONL: Outer nuclear layer, EZ: Ellipsoid zone, RPE: Retinal pigment epithelium

Figure 5:

Retinal abnormality resulting in false positive toxicity detection. (A) EZ-RPE thickness maps demonstrating some perifoveal EZ integrity loss including a significant area of focal loss. (B) B-scan in the area of focal loss demonstrates an isolated pocket of SRF that has resulted in EZ loss rather than a secondary loss due to hydroxychloroquine toxicity (layer segmentation of the ILM (yellow line), ONL (green line), EZ (red line) and RPE (orange line)). OCT: Optical coherence tomography, ILM: Inner limiting membrane, ONL: Outer nuclear layer, EZ: Ellipsoid zone, RPE: Retinal pigment epithelium