Assessing the performances of a chatbot to collect real-life data of patients suffering from primary headache disorders

Abstract

Background: There are many scales for screening the impact of a disease. These scales are generally used to diagnose or assess the type and severity of a disease and are carried out by doctors. The chatbot helps patients suffering from primary headache disorders through personalized text messages. It could be used to collect patient-reported outcomes.

Objective: The aims of this study were (1) to study whether the collection and analysis of remote scores, without prior medical intervention, are possible by a chatbot, (2) to perform suggested diagnosis and define the type of headaches, and (3) to assess the patient satisfaction and engagement with the chatbot.

Method: Voluntary users of the chatbot were recruited online. They had to be over 18 and have a personal history of headaches. A questionnaire was presented (1) by text messages to the participants to evaluate migraines (2) based on the criteria of the International Headache Society. Then, the Likert scale (3) was used to assess overall satisfaction with the use of the chatbot.

Results: We included 610 participants with primary headache disorders. A total of 89.94% (572/610) participants had fully completed the questionnaire (eight items), 4.72% (30/610) had partially completed it, and 5.41% (33) had refused to complete it. Statistical analysis was performed on 86.01% (547/610) of participants. Auto diagnostic showed that 14.26% (78/547) participants had a tension headache, and 85.74% (469/547) had a probable migraine. In this population, 15.78% (74/469) suffered from migraine without probable aura, and 84.22% (395/469) had migraine without aura. The patient's age had a significant incidence regarding the auto diagnosis (P = .008<.05). The evaluation of overall satisfaction shows that a total of 93.9% (599/610) of users were satisfied or very satisfied regarding the timeliness of responses the chatbot provides.

Conclusion: The study confirmed that it was possible to obtain such a collection remotely, and quickly (average time of 3.24 min) with a high success rate (89.67% (547/610) participants who had fully completed the IHS questionnaire). Users were strongly engaged through chatbot: out of the total number of participants, we observed a very low number of uncompleted questionnaires (6.23% (38/610)). Conversational agents can be used to remotely collect data on the nature of the symptoms of patients suffering from primary headache disorders. These results are promising regarding patient engagement and trust in the chatbot.

Keywords: Chatbot; conversational agent; patient-reported outcomes.

Figure 1.

The chatbot's interface.

Figure 2.

Overall satisfaction rating.