AI-based assessment of Clinical Activity Score and detection of active thyroid eye disease using facial images: validation of Glandy CAS

Abstract

Purpose: The Clinical Activity Score (CAS) is widely used to assess thyroid eye disease (TED) activity but can vary based on the evaluator's expertise. We developed and externally validated Glandy CAS, a machine learning (ML)-assisted system for detecting active TED (CAS ≥3) using digital facial images. This clinical trial aimed to gain approval from the Korea Ministry of Food and Drug Safety (KMFDS) for this Software as a Medical Device (SaMD).

Methods: This is a clinical trial based on the retrospective cohort. Glandy CAS analysed 756 photos of patients with TED, classifying them as having active or inactive TED. Its diagnostic performance was compared with that of three general ophthalmologists (less than 5 years of experience), using the F1 score. The reference CAS was determined by an oculoplastic specialist.

Results: Active TED was detected in 207 of 756 patients. Glandy CAS achieved a sensitivity of 87.9%, specificity of 95.8% and an F1 score of 0.88. In comparison, general ophthalmologists had a sensitivity of 60.4%, specificity of 83.0% and an F1 score of 0.57. Glandy CAS predicted CAS within 1 point of the reference score in 82.3% of cases, with a mean absolute error of 0.83.

Conclusions: Glandy CAS, an ML-assisted system for detecting active TED using facial images, showed high accuracy and outperformed general ophthalmologists. This system can consistently and accurately assess disease activity, facilitating early detection and timely treatment of active TED. Based on this clinical trial, the SaMD received KMFDS approval (Product Licence No., 24-93).

Keywords: Clinical Trial; Diagnostic tests/Investigation; Orbit; Smartphone.

Figure 1. Image preprocessing module. The preprocessing module extracts diagnostic features from input images (A) in three steps: (B) face detection and cropping to isolate the ocular region, (C) segmentation of the eye and corneal regions, (D) generation of region-specific model inputs and preparation of image patches for sign-specific classifiers. This process enables the system to focus on relevant anatomical areas associated with each CAS-related sign. All facial images used in this figure are synthetically generated and do not depict real individuals. CAS, Clinical Activity Score.

Figure 2. Structural diagram of ML-assisted system to predict active TED. AI, artificial intelligence; CAS, Clinical Activity Score; ML, machine learning; TED, thyroid eye disease.