Assessing the Usage and Usability of a Mental Health Advice Telephone Service in Uganda: Mixed Methods Study

Abstract

Background: Harnessing mobile health (mHealth) solutions could improve the delivery of mental health services and mitigate their impact in Uganda and similar low-resource settings. However, successful adoption requires that mHealth solutions have good usability. We have previously implemented a telephone service to provide mental health information and advice in English and Luganda, utilizing an automated interactive voice response (IVR) system linked to live agents, including mental health care workers and peer support workers.

Objective: This study aims to assess the usage and usability of this mental health telephone service.

Methods: We obtained usage data from the system's call logs over 18 months to study call volumes and trends. We then surveyed callers to gather their characteristics and assess usability using the Telehealth Usability Questionnaire. Additionally, call recordings were evaluated for conversation quality by 3 independent health care professionals, using the Telephone Nursing Dialogue Process, and correlations between quality and usability aspects were investigated.

Results: Over 18 months, the system received 2863 meaningful calls (ie, calls that went past the welcome message) from 1125 unique telephone numbers. Of these, 1153 calls (40.27%) stopped at the prerecorded IVR information, while 1710 calls (59.73%) opted to speak to an agent. Among those who chose to speak with an agent, 1292 calls (75.56%) were answered, 393 calls (22.98%) went to voicemail and were returned in the following working days, and 25 calls (1.46%) were not answered. Usage was generally sustained over time, with spikes in call volume corresponding to marketing events. The survey (n=240) revealed that most callers were caregivers of patients with mental health issues (n=144, 60.0%) or members of the general public (n=46, 19.2%), while a few were patients with mental health issues (n=44, 18.3%). Additionally, the majority were male (n=143, 59.6%), spoke English (n=180, 75.0%), had postsecondary education (n=164, 68.3%), lived within 1 hour or less from Butabika Hospital (n=187, 77.9%), and were aged 25-44 years (n=160, 66.7%). The overall usability score for the system was 4.12 on a 5-point scale, significantly higher than the recommended target usability score of 4 (P=.006). The mean scores for usability components ranged from 3.66 for reliability to 4.41 for ease of use, with all components, except reliability, scoring higher than 4 or falling within its CI. Usability scores were higher for Luganda speakers compared with English speakers, but there was no association with other participant characteristics such as sex, distance from the hospital, age, marital status, duration of symptoms, or treatment status. The quality of call conversations (n=50) was rated at 4.35 out of 5 and showed a significant correlation with usability (Pearson r=0.34, P=.02).

Conclusions: We found sustained usage of the mental health telephone service, along with a positive user experience and high satisfaction across various user characteristics. mHealth solutions like this should be embraced and replicated to enhance the delivery of health services in Uganda and similar low-resource settings.

Keywords: Africa; Uganda; evaluation; mHealth; mental health; mixed method; satisfaction; telephone service; usability.

Figure 1

Mental health messages listened to in the IVR. IVR: interactive voice response.

Figure 2

Distribution of call types per day of the week. There were no live agents on Saturdays and Sundays. IVR: interactive voice response; VMS: voicemail system.