Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Sep 2;7(9):e24377.
doi: 10.2196/24377.

Designing a Clinical Decision Support Tool That Leverages Machine Learning for Suicide Risk Prediction: Development Study in Partnership With Native American Care Providers

Emily E Haroz  1 , Fiona Grubin  1 , Novalene Goklish  1 , Shardai Pioche  1 , Mary Cwik  1 , Allison Barlow  1 , Emma Waugh  1 , Jason Usher  1 , Matthew C Lenert  2 , Colin G Walsh  2   3   4
Affiliations

Designing a Clinical Decision Support Tool That Leverages Machine Learning for Suicide Risk Prediction: Development Study in Partnership With Native American Care Providers

Emily E Haroz et al. JMIR Public Health Surveill. .

Abstract

Background: Machine learning algorithms for suicide risk prediction have been developed with notable improvements in accuracy. Implementing these algorithms to enhance clinical care and reduce suicide has not been well studied.

Objective: This study aims to design a clinical decision support tool and appropriate care pathways for community-based suicide surveillance and case management systems operating on Native American reservations.

Methods: Participants included Native American case managers and supervisors (N=9) who worked on suicide surveillance and case management programs on 2 Native American reservations. We used in-depth interviews to understand how case managers think about and respond to suicide risk. The results from interviews informed a draft clinical decision support tool, which was then reviewed with supervisors and combined with appropriate care pathways.

Results: Case managers reported acceptance of risk flags based on a predictive algorithm in their surveillance system tools, particularly if the information was available in a timely manner and used in conjunction with their clinical judgment. Implementation of risk flags needed to be programmed on a dichotomous basis, so the algorithm could produce output indicating high versus low risk. To dichotomize the continuous predicted probabilities, we developed a cutoff point that favored specificity, with the understanding that case managers' clinical judgment would help increase sensitivity.

Conclusions: Suicide risk prediction algorithms show promise, but implementation to guide clinical care remains relatively elusive. Our study demonstrates the utility of working with partners to develop and guide the operationalization of risk prediction algorithms to enhance clinical care in a community setting.

Keywords: Native American health; implementation; machine learning; suicide prevention.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest: None declared.

Figures

Figure 1
Figure 1
Factors influencing case manager planning and representative quotes.
Figure 2
Figure 2
Responses to long-term suicide risk.
Figure 3
Figure 3
Clinical decision and appropriate care pathways tool. Pink form is the name for the follow-up form that is used at the follow-up visit.
See this image and copyright information in PMC

References

    1. Cwik MF, Barlow A, Goklish N, Larzelere-Hinton F, Tingey L, Craig M, Lupe R, Walkup J. Community-based surveillance and case management for suicide prevention: an American Indian tribally initiated system. Am J Public Health. 2014 Jun;104(S3):18–23. doi: 10.2105/ajph.2014.301872. - DOI - PMC - PubMed
    1. Riblet NB, Shiner B, Young-Xu Y, Watts BV. Strategies to prevent death by suicide: meta-analysis of randomised controlled trials. Br J Psychiatry. 2017 Jun 02;210(6):396–402. doi: 10.1192/bjp.bp.116.187799.S0007125000281646 - DOI - PubMed
    1. Cwik M, Goklish N, Masten K, Lee A, Suttle R, Alchesay M, O'Keefe V, Barlow A. "Let our Apache Heritage and Culture Live on Forever and Teach the Young Ones": Development of the elders' resilience curriculum, an upstream suicide prevention approach for American Indian youth. Am J Community Psychol. 2019 Sep 16;64(1-2):137–45. doi: 10.1002/ajcp.12351. - DOI - PubMed
    1. Cwik M, Tingey L, Lee A, Suttle R, Lake K, Walkup JT, Barlow A. Development and piloting of a brief intervention for suicidal American Indian adolescents. Am Indian Alsk Native Ment Health Res. 2016;23(1):105–24. doi: 10.5820/aian.2301.2016.105. - DOI - PubMed
    1. Cwik MF, Tingey L, Maschino A, Goklish N, Larzelere-Hinton F, Walkup J, Barlow A. Decreases in suicide deaths and attempts linked to the white mountain apache suicide surveillance and prevention system, 2001–2012. Am J Public Health. 2016 Dec;106(12):2183–9. doi: 10.2105/ajph.2016.303453. - DOI - PMC - PubMed

Publication types

LinkOut - more resources