A diagnostic support system based on pain drawings: binary and k-disease classification of EDS, GBS, FSHD, PROMM, and a control group with Pain2D

Abstract

Background and objective: The diagnosis of rare diseases (RDs) is often challenging due to their rarity, variability and the high number of individual RDs, resulting in a delay in diagnosis with adverse effects for patients and healthcare systems. The development of computer assisted diagnostic decision support systems could help to improve these problems by supporting differential diagnosis and by prompting physicians to initiate the right diagnostic tests. Towards this end, we developed, trained and tested a machine learning model implemented as part of the software called Pain2D to classify four rare diseases (EDS, GBS, FSHD and PROMM), as well as a control group of unspecific chronic pain, from pen-and-paper pain drawings filled in by patients.

Methods: Pain drawings (PDs) were collected from patients suffering from one of the four RDs, or from unspecific chronic pain. The latter PDs were used as an outgroup in order to test how Pain2D handles more common pain causes. A total of 262 (59 EDS, 29 GBS, 35 FSHD, 89 PROMM, 50 unspecific chronic pain) PDs were collected and used to generate disease specific pain profiles. PDs were then classified by Pain2D in a leave-one-out-cross-validation approach.

Results: Pain2D was able to classify the four rare diseases with an accuracy of 61-77% with its binary classifier. EDS, GBS and FSHD were classified correctly by the Pain2D k-disease classifier with sensitivities between 63 and 86% and specificities between 81 and 89%. For PROMM, the k-disease classifier achieved a sensitivity of 51% and specificity of 90%.

Conclusions: Pain2D is a scalable, open-source tool that could potentially be trained for all diseases presenting with pain.

Keywords: AI; Diagnostic support; Machine learning; ORPHA: 2103; ORPHA: 269; ORPHA: 287; ORPHA: 606; Pain drawings; Rare diseases; k-disease classification.

Fig. 1

Pain profiles of the five diagnostic groups used in this study, EDS (A), GBS (B), FSHD (C), PROMM (D), chronic pain (CP, E) and RARE (F). The depicted pain profiles were constructed by Pain2D from 29 EDS (A), 59 GBS (B), 35 FSHD (C), 89 PROMM (D) and 50 CP (E) PDs. RARE is based on 29 EDS, 59 GBS, 35 FSHD and 89 PROMM PDs (F)

Fig. 2

ROC curve for classification of PDs into RARE and CG with the binary classifier of Pain2D. The light blue area indicates the 95% confidence interval. Blue crosshairs indicate optimal classification threshold of 0.41

Fig. 3

Receiver operating characteristics (ROC) curves for binary classification of each RD versus CP. A ROC curve binary classification of EDS and CP. AUC = 0.899 (CI 0.829–0.954), B ROC curve binary classification of GBS and CP. AUC = 0.921 (CI 0.853–0.973), C ROC curve binary classification of FSHD or CP. AUC = 0.854 (CI 0.770–0.930), D ROC curve binary classification of PROMM and CP. AUC = 0.846 (CI 0.774–0.908). Confidence intervals are depicted as light blue band. Blue crosshairs indicate optimal threshold for classification