Technology-Enabled Solutions for Australian Mental Health Services Reform: Impact Evaluation

Abstract

Background: Health information technologies (HITs) are becoming increasingly recognized for their potential to provide innovative solutions to improve the delivery of mental health services and drive system reforms for better outcomes.

Objective: This paper describes the baseline results of a study designed to systematically monitor and evaluate the impact of implementing an HIT, namely the InnoWell Platform, into Australian mental health services to facilitate the iterative refinement of the HIT and the service model in which it is embedded to meet the needs of consumers and their supportive others as well as health professionals and service providers.

Methods: Data were collected via web-based surveys, semistructured interviews, and a workshop with staff from the mental health services implementing the InnoWell Platform to systematically monitor and evaluate its impact. Descriptive statistics, Fisher exact tests, and a reliability analysis were used to characterize the findings from the web-based surveys, including variability in the results between the services. Semistructured interviews were coded using a thematic analysis, and workshop data were coded using a basic content analysis.

Results: Baseline data were collected from the staff of 3 primary youth mental health services (n=18), a counseling service for veterans and their families (n=23), and a helpline for consumers affected by eating disorders and negative body image issues (n=6). As reported via web-based surveys, staff members across the services consistently agreed or strongly agreed that there was benefit associated with using technology as part of their work (38/47, 81%) and that the InnoWell Platform had the potential to improve outcomes for consumers (27/45, 60%); however, there was less certainty as to whether their consumers' capability to use technology aligned with how the InnoWell Platform would be used as part of their mental health care (11/45, 24% of the participants strongly disagreed or disagreed; 15/45, 33% were neutral; and 19/45, 42% strongly agreed or agreed). During the semistructured interviews (n=3) and workshop, participants consistently indicated that the InnoWell Platform was appropriate for their respective services; however, they questioned whether the services' respective consumers had the digital literacy required to use the technology. Additional potential barriers to implementation included health professionals' digital literacy and service readiness for change.

Conclusions: Despite agreement among participants that HITs have the potential to result in improved outcomes for consumers and services, service readiness for change (eg, existing technology infrastructure and the digital literacy of staff and consumers) was noted to potentially impact the success of implementation, with less than half (20/45, 44%) of the participants indicating that their service was ready to implement new technologies to enhance mental health care. Furthermore, participants reported mixed opinions as to whether it was their responsibility to recommend technology as part of standard care.

Keywords: community-based participatory research; evaluation methodology; health care reform; mental health; mobile phone; technology.

Figure 1

Codes and themes.