Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Randomized Controlled Trial
. 2021 Apr 13;23(1):112.
doi: 10.1186/s13075-021-02498-8.

Accuracy, patient-perceived usability, and acceptance of two symptom checkers (Ada and Rheport) in rheumatology: interim results from a randomized controlled crossover trial

Johannes Knitza  1   2   3 , Jacob Mohn  4   5 , Christina Bergmann  4   5 , Eleni Kampylafka  4   5 , Melanie Hagen  4   5 , Daniela Bohr  4   5 , Harriet Morf  4   5 , Elizabeth Araujo  4   5 , Matthias Englbrecht  4   5 , David Simon  4   5 , Arnd Kleyer  4   5 , Timo Meinderink  4   5 , Wolfgang Vorbrüggen  6   7 , Cay Benedikt von der Decken  7   8   9 , Stefan Kleinert  7   10 , Andreas Ramming  4   5 , Jörg H W Distler  4   5 , Nicolas Vuillerme  11   12   13 , Achim Fricker  14 , Peter Bartz-Bazzanella  7   9 , Georg Schett  4   5 , Axel J Hueber #  4   15 , Martin Welcker #  7   16
Affiliations
Randomized Controlled Trial

Accuracy, patient-perceived usability, and acceptance of two symptom checkers (Ada and Rheport) in rheumatology: interim results from a randomized controlled crossover trial

Johannes Knitza et al. Arthritis Res Ther. .

Abstract

Background: Timely diagnosis and treatment are essential in the effective management of inflammatory rheumatic diseases (IRDs). Symptom checkers (SCs) promise to accelerate diagnosis, reduce misdiagnoses, and guide patients more effectively through the health care system. Although SCs are increasingly used, there exists little supporting evidence.

Objective: To assess the diagnostic accuracy, patient-perceived usability, and acceptance of two SCs: (1) Ada and (2) Rheport.

Methods: Patients newly presenting to a German secondary rheumatology outpatient clinic were randomly assigned in a 1:1 ratio to complete Ada or Rheport and consecutively the respective other SCs in a prospective non-blinded controlled randomized crossover trial. The primary outcome was the accuracy of the SCs regarding the diagnosis of an IRD compared to the physicians' diagnosis as the gold standard. The secondary outcomes were patient-perceived usability, acceptance, and time to complete the SC.

Results: In this interim analysis, the first 164 patients who completed the study were analyzed. 32.9% (54/164) of the study subjects were diagnosed with an IRD. Rheport showed a sensitivity of 53.7% and a specificity of 51.8% for IRDs. Ada's top 1 (D1) and top 5 disease suggestions (D5) showed a sensitivity of 42.6% and 53.7% and a specificity of 63.6% and 54.5% concerning IRDs, respectively. The correct diagnosis of the IRD patients was within the Ada D1 and D5 suggestions in 16.7% (9/54) and 25.9% (14/54), respectively. The median System Usability Scale (SUS) score of Ada and Rheport was 75.0/100 and 77.5/100, respectively. The median completion time for both Ada and Rheport was 7.0 and 8.5 min, respectively. Sixty-four percent and 67.1% would recommend using Ada and Rheport to friends and other patients, respectively.

Conclusions: While SCs are well accepted among patients, their diagnostic accuracy is limited to date.

Trial registration: DRKS.de, DRKS00017642 . Registered on 23 July 2019.

Keywords: Acceptability; Accuracy; Apps; Diagnosis; Rheumatology; Symptom checker; Usability; eHealth.

PubMed Disclaimer

Conflict of interest statement

JK has received research support from Novartis Pharma GmbH. Qinum and RheumaDatenRhePort developed and hold rights for Rheport. WV, CD, SK, PB, and MW are members of RheumaDatenRhePort. AF, WV, CD, and PB were involved in the development of Rheport. JK is a member of the scientific board of RheumaDatenRhePort.

Figures

Fig. 1
Fig. 1
Screenshots of the Ada and Rheport symptom checker. 1The German version of Ada was used in the study. 2The Rheport menu was translated into English for this figure
Fig. 2
Fig. 2
Patient flow diagram
See this image and copyright information in PMC

References

    1. Combe B, Landewe R, Daien CI, Hua C, Aletaha D, Álvaro-Gracia JM, Bakkers M, Brodin N, Burmester GR, Codreanu C, Conway R, Dougados M, Emery P, Ferraccioli G, Fonseca J, Raza K, Silva-Fernández L, Smolen JS, Skingle D, Szekanecz Z, Kvien TK, van der Helm-van Mil A, van Vollenhoven R. 2016 update of the EULAR recommendations for the management of early arthritis. Ann Rheum Dis. 2017;76(6):948–959. doi: 10.1136/annrheumdis-2016-210602. - DOI - PubMed
    1. Quinn MA, Emery P. Window of opportunity in early rheumatoid arthritis: possibility of altering the disease process with early intervention. Clin Exp Rheumatol. 2003;21(0392-856X (Print)):154–157. - PubMed
    1. Villeneuve E, Nam JL, Bell MJ, Deighton CM, Felson DT, Hazes JM, IB MI, Silman AJ, Solomon DH, Thompson AE, White PHP, et al. A systematic literature review of strategies promoting early referral and reducing delays in the diagnosis and management of inflammatory arthritis. Ann Rheum Dis. 2012;72(1468–2060 (Electronic)):13–22. - PubMed
    1. Benesova K, Lorenz HM, Lion V, Voigt A, Krause A, Sander O, Schneider M, Feuchtenberger M, Nigg A, Leipe J, Briem S, Tiessen E, Haas F, Rihl M, Meyer-Olson D, Baraliakos X, Braun J, Schwarting A, Dreher M, Witte T, Assmann G, Hoeper K, Schmidt RE, Bartz-Bazzanella P, Gaubitz M, Specker C. Früh- und Screeningsprechstunden: Ein notwendiger Weg zur besseren Frühversorgung in der internistischen Rheumatologie? Z Rheumatol. 2019;78(8):722–742. doi: 10.1007/s00393-019-0683-y. - DOI - PubMed
    1. Raza K, Stack R, Kumar K, Filer A, Detert J, Bastian H, Burmester GR, Sidiropoulos P, Kteniadaki E, Repa A, Saxne T, Turesson C, Mann H, Vencovsky J, Catrina A, Chatzidionysiou A, Hensvold A, Rantapää-Dahlqvist S, Binder A, Machold K, Kwiakowska B, Ciurea A, Tamborrini G, Kyburz D, Buckley CD. Delays in assessment of patients with rheumatoid arthritis: variations across Europe. Ann Rheum Dis. 2011;70(10):1822–1825. doi: 10.1136/ard.2011.151902. - DOI - PubMed

Publication types

LinkOut - more resources