Can a decision support system accelerate rare disease diagnosis? Evaluating the potential impact of Ada DX in a retrospective study

Abstract

Background: Rare disease diagnosis is often delayed by years. A primary factor for this delay is a lack of knowledge and awareness regarding rare diseases. Probabilistic diagnostic decision support systems (DDSSs) have the potential to accelerate rare disease diagnosis by suggesting differential diagnoses for physicians based on case input and incorporated medical knowledge. We examine the DDSS prototype Ada DX and assess its potential to provide accurate rare disease suggestions early in the course of rare disease cases.

Results: Ada DX suggested the correct disease earlier than the time of clinical diagnosis among the top five fit disease suggestions in 53.8% of cases (50 of 93), and as the top fit disease suggestion in 37.6% of cases (35 of 93). The median advantage of correct disease suggestions compared to the time of clinical diagnosis was 3 months or 50% for top five fit and 1 month or 21% for top fit. The correct diagnosis was suggested at the first documented patient visit in 33.3% (top 5 fit), and 16.1% of cases (top fit), respectively. Wilcoxon signed-rank test shows a significant difference between the time to clinical diagnosis and the time to correct disease suggestion for both top five fit and top fit (z-score -6.68, respective -5.71, α=0.05, p-value <0.001).

Conclusion: Ada DX provided accurate rare disease suggestions in most rare disease cases. In many cases, Ada DX provided correct rare disease suggestions early in the course of the disease, sometimes at the very beginning of a patient journey. The interpretation of these results indicates that Ada DX has the potential to suggest rare diseases to physicians early in the course of a case. Limitations of this study derive from its retrospective and unblinded design, data input by a single user, and the optimization of the knowledge base during the course of the study. Results pertaining to the system's accuracy should be interpreted cautiously. Whether the use of Ada DX reduces the time to diagnosis in rare diseases in a clinical setting should be validated in prospective studies.

Keywords: Ada DX; Artificial intelligence; Diagnostic decision support system; Probabilistic reasoning; Rare disease diagnosis; Time to diagnosis.

Fig. 1

Ada DX. Screenshot of the DDSS research prototype Ada DX showing a case of a patient with TRAPS. The later confirmed diagnosis was suggested by Ada DX in an early visit. Top: Basic patient data and case timeline. Left: Symptom search and suggested symptoms. Center: Entered symptoms with their attributes, green contribution lines, selected diseases, bars visualizing disease probability (green) and fit (purple). Right: Lists of differential diagnoses ranked by ’probability’ and ’fit’ and links to similar cases

Fig. 2

Distribution of TD, TF and T5F. Boxplots for time to clinical diagnosis (TD), time to correct top fit suggestion (TF) and time to correct top 5 fit suggestion (T5F). Outliers outside the whiskers were cut out. Additional information is provided in Table 3

Fig. 3

Distribution of TF/TD by TD. Visualisation of TF relative to TD, grouped by TD. Number of cases per group: 0m: 5; 1-12m: 33; 1-5y: 30; >5y: 25

Fig. 4

Distribution of T5F/TD by TD. Visualisation of T5F relative to TD, grouped by TD. Number of cases per group: 0m: 5; 1-12m: 33; 1-5y: 30; >5y: 25