Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Meta-Analysis
. 2021 Jul 27;7(7):CD012944.
doi: 10.1002/14651858.CD012944.pub2.

Decision-support tools via mobile devices to improve quality of care in primary healthcare settings

Smisha Agarwal  1 , Claire Glenton  2 , Tigest Tamrat  3 , Nicholas Henschke  4 , Nicola Maayan  5 , Marita S Fønhus  2 , Garrett L Mehl  3 , Simon Lewin  2   6
Affiliations
Meta-Analysis

Decision-support tools via mobile devices to improve quality of care in primary healthcare settings

Smisha Agarwal et al. Cochrane Database Syst Rev. .

Abstract

Background: The ubiquity of mobile devices has made it possible for clinical decision-support systems (CDSS) to become available to healthcare providers on handheld devices at the point-of-care, including in low- and middle-income countries. The use of CDSS by providers can potentially improve adherence to treatment protocols and patient outcomes. However, the evidence on the effect of the use of CDSS on mobile devices needs to be synthesized. This review was carried out to support a World Health Organization (WHO) guideline that aimed to inform investments on the use of decision-support tools on digital devices to strengthen primary healthcare.

Objectives: To assess the effects of digital clinical decision-support systems (CDSS) accessible via mobile devices by primary healthcare providers in the context of primary care settings.

Search methods: We searched CENTRAL, MEDLINE, Embase, Global Index Medicus, POPLINE, and two trial registries from 1 January 2000 to 9 October 2020. We conducted a grey literature search using mHealthevidence.org and issued a call for papers through popular digital health communities of practice. Finally, we conducted citation searches of included studies.

Selection criteria: Study design: we included randomized trials, including full-text studies, conference abstracts, and unpublished data irrespective of publication status or language of publication. Types of participants: we included studies of all cadres of healthcare providers, including lay health workers and other individuals (administrative, managerial, and supervisory staff) involved in the delivery of primary healthcare services using clinical decision-support tools; and studies of clients or patients receiving care from primary healthcare providers using digital decision-support tools. Types of interventions: we included studies comparing digital CDSS accessible via mobile devices with non-digital CDSS or no intervention, in the context of primary care. CDSS could include clinical protocols, checklists, and other job-aids which supported risk prioritization of patients. Mobile devices included mobile phones of any type (but not analogue landline telephones), as well as tablets, personal digital assistants, and smartphones. We excluded studies where digital CDSS were used on laptops or integrated with electronic medical records or other types of longitudinal tracking of clients.

Data collection and analysis: A machine learning classifier that gave each record a probability score of being a randomized trial screened all search results. Two review authors screened titles and abstracts of studies with more than 10% probability of being a randomized trial, and one review author screened those with less than 10% probability of being a randomized trial. We followed standard methodological procedures expected by Cochrane and the Effective Practice and Organisation of Care group. We used the GRADE approach to assess the certainty of the evidence for the most important outcomes.

Main results: Eight randomized trials across varying healthcare contexts in the USA,. India, China, Guatemala, Ghana, and Kenya, met our inclusion criteria. A range of healthcare providers (facility and community-based, formally trained, and lay workers) used digital CDSS. Care was provided for the management of specific conditions such as cardiovascular disease, gastrointestinal risk assessment, and maternal and child health. The certainty of evidence ranged from very low to moderate, and we often downgraded evidence for risk of bias and imprecision. We are uncertain of the effect of this intervention on providers' adherence to recommended practice due to the very low certainty evidence (2 studies, 185 participants). The effect of the intervention on patients' and clients' health behaviours such as smoking and treatment adherence is mixed, with substantial variation across outcomes for similar types of behaviour (2 studies, 2262 participants). The intervention probably makes little or no difference to smoking rates among people at risk of cardiovascular disease but probably increases other types of desired behaviour among patients, such as adherence to treatment. The effect of the intervention on patients'/clients' health status and well-being is also mixed (5 studies, 69,767 participants). It probably makes little or no difference to some types of health outcomes, but we are uncertain about other health outcomes, including maternal and neonatal deaths, due to very low-certainty evidence. The intervention may slightly improve patient or client acceptability and satisfaction (1 study, 187 participants). We found no studies that reported the time between the presentation of an illness and appropriate management, provider acceptability or satisfaction, resource use, or unintended consequences.

Authors' conclusions: We are uncertain about the effectiveness of mobile phone-based decision-support tools on several outcomes, including adherence to recommended practice. None of the studies had a quality of care framework and focused only on specific health areas. We need well-designed research that takes a systems lens to assess these issues.

PubMed Disclaimer

Conflict of interest statement

SA: The author was commissioned by the WHO to conduct this review.

CG: I am an editor for Cochrane Effective Practice and Organisation of Care. I was not involved in the editorial process for this review.

TT: Works for the WHO.

NH: Since June 2016, I have been employed by Cochrane Response, an evidence services unit operated by Cochrane. Cochrane Response was contracted by the WHO to produce this review.

MS: None known.

GM: Owns stock in Apple computers.

NM: I previously worked for Enhanced Reviews Ltd, a company that conducts systematic reviews, mostly for the public sector. Since June 2016, I have been employed by Cochrane Response, an evidence services unit operated by Cochrane. Cochrane Response was contracted by the World Health Organization to produce this review.

MF: The author was commissioned by the WHO to conduct this review.

GM: Works for the WHO.

SL: I am the Joint Co‐ordinating Editor for Cochrane Effective Practice and Organisation of Care. I was not involved in the editorial process for this review.

Figures

1
1
Study flow diagram.
2
2
Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.
3
3
Risk of bias summary: review authors' judgements about each risk of bias item for each included study.
1.2
1.2. Analysis
Comparison 1: Mobile clinical decision support compared to standard care for patients, Outcome 2: Patients' health status and well‐being (dichotomous outcomes)
1.3
1.3. Analysis
Comparison 1: Mobile clinical decision support compared to standard care for patients, Outcome 3: Patients' health status and well‐being (continuous outcomes)
1.4
1.4. Analysis
Comparison 1: Mobile clinical decision support compared to standard care for patients, Outcome 4: Patients' health status and well‐being (dichotomous undesirable outcome)
1.5
1.5. Analysis
Comparison 1: Mobile clinical decision support compared to standard care for patients, Outcome 5: Patients' health status and well‐being (continuous undesirable outcomes)
1.6
1.6. Analysis
Comparison 1: Mobile clinical decision support compared to standard care for patients, Outcome 6: Patients' health status and well‐being (neonatal deaths)
1.7
1.7. Analysis
Comparison 1: Mobile clinical decision support compared to standard care for patients, Outcome 7: Patients' acceptability and satisfaction
See this image and copyright information in PMC

Update of

References

References to studies included in this review

Amoakoh 2019 {published data only}
    1. Amoakoh HB, Klipstein-Grobusch K, Agyepong IA, Zuithoff NP, Amoakoh-Coleman M, Kayode GA, et al. The effect of an mHealth clinical decision-making support system on neonatal mortality in a low resource setting: a cluster-randomized controlled trial. EClinicalMedicine 2019;12:31-42. - PMC - PubMed
Berner 2006 {published data only}
    1. Berner ES, Houston TK, Ray MN, Allison JJ, Heudebert GR, Chatham WW, et al. Improving ambulatory prescribing safety with a handheld decision support system: a randomized controlled trial. Journal of the American Medical Informatics Association 2006;13(2):171-9. - PMC - PubMed
Eaton 2011 {published data only}
    1. Eaton CB, Parker DR, Borkan J, McMurray J, Roberts MB, Lu B, et al. Translating cholesterol guidelines into primary care practice: a multimodal cluster randomized trial. Annals of Family Medicine 2011;9(6):528-37. - PMC - PubMed
Gautham 2015 {published data only}
    1. Gautham M, Iyengar MS, Johnson CW. Mobile phone–based clinical guidance for rural health providers in India. Health informatics journal 2015;21(4):253-66. - PubMed
Heisler 2014 {published data only}
    1. Heisler M, Choi H, Palmisano G, Mase R, Richardson C, Fagerlin A, et al. Comparison of community health worker–led diabetes medication decision-making support for low-income Latino and African American adults with diabetes using e-health tools versus print materials: a randomized, controlled trial. Annals of Internal Medicine 2014;161(10 Suppl):S13-22. - PMC - PubMed
Keane 2018 {published data only}
    1. Frank T, Keane E, Roschnik N, Emary C, O'Leary M, Snyder L. Developing a mobile healthapp to manage acute malnutrition: a five-country experience. Field Exchange 2017;54:7.
    1. Keane E, Roschnik N, Chui J, Osman IA, Osman HM. Evaluation of mobile application to support the treatment of acutely malnourished children in Wajir county, Kenya. Field Exchange 2018;57:61.
Martinez 2018 {published data only}
    1. Martinez B, Ixen EC, Hall-Clifford R, Juarez M, Miller AC, Francis A, et al. mHealth intervention to improve the continuum of maternal and perinatal care in rural Guatemala: a pragmatic, randomized controlled feasibility trial. Reproductive health 2018;15(1):120. - PMC - PubMed
Tian 2015 {published data only}
    1. Tian M, Ajay V, Dunzhu D, Hameed S, Li X, Liu Z, et al. A cluster-randomized controlled trial of a simplified multifaceted management program for individuals at high cardiovascular risk (SimCard Trial) in Rural Tibet, China, and Haryana, India. Circulation 2015;17:115. - PMC - PubMed

References to studies excluded from this review

Arts 2017 {published data only}
    1. Arts DL, Abu-Hanna A, Medlock SK, Weert HC. Effectiveness and usage of a decision support system to improve stroke prevention in general practice: a cluster randomized controlled trial. PloS One 2017;12(2):e0170974. - PMC - PubMed
Caballero‐Ruiz 2017 {published data only}
    1. Caballero-Ruiz E, García-Sáez G, Rigla M, Villaplana M, Pons B, Hernando ME. A web-based clinical decision support system for gestational diabetes: automatic diet prescription and detection of insulin needs. International Journal of Medical Informatics 2017;102:35-49. - PubMed
Carroll 2012 {published data only}
    1. Carroll AE, Biondich P, Anand V, Dugan TM, Downs SM. A randomized controlled trial of screening for maternal depression with a clinical decision support system. Journal of the American Medical Informatics Association 2012;20(2):311-6. - PMC - PubMed
Carroll 2018 {published data only}
    1. Carroll JK, Pulver G, Dickinson LM, Pace WD, Vassalotti JA, Kimminau KS, et al. Effect of 2 clinical decision support strategies on chronic kidney disease outcomes in primary care: a cluster randomized trial. JAMA Network Open 2018;1(6):e183377. - PMC - PubMed
Field 2009 {published data only}
    1. Field TS, Rochon P, Lee M, Gavendo L, Baril JL, Gurwitz JH. Computerized clinical decision support during medication ordering for long-term care residents with renal insufficiency. Journal of the American Medical Informatics Association 2009;16(4):480-5. - PMC - PubMed
Forrest 2013 {published data only}
    1. Forrest CB, Fiks AG, Bailey LC, Localio R, Grundmeier RW, Richards T, et al. Improving adherence to otitis media guidelines with clinical decision support and physician feedback. Pediatrics 2013;131(4):e1071-81. - PubMed
Gill 2011 {published data only}
    1. Gill JM, Mainous AG, Koopman RJ, Player MS, Everett CJ, Chen YX, et al. Impact of EHR-based clinical decision support on adherence to guidelines for patients on NSAIDs: a randomized controlled trial. Annals of Family Medicine 2011;9(1):22-30. - PMC - PubMed
Gong 2019 {published data only}
    1. Gong E, Gu W, Sun C, Turner EL, Zhou Y, Li Z, et al. System-integrated technology-enabled model of care to improve the health of stroke patients in rural China: protocol for SINEMA – a cluster-randomized controlled trial. American Heart Journal 2019;207:27-39. - PubMed
Gulliford 2019 {published data only}
    1. Gulliford MC, Prevost AT, Charlton J, Juszczyk D, Soames J, McDermott L, et al. Effectiveness and safety of electronically delivered prescribing feedback and decision support on antibiotic use for respiratory illness in primary care: REDUCE cluster randomised trial. BMJ 2019;364:l236. - PMC - PubMed
Holbrook 2011 {published data only}
    1. Holbrook A, Pullenayegum E, Thabane L, Troyan S, Foster G, Keshavjee K, et al. Shared electronic vascular risk decision support in primary care: Computerization of Medical Practices for the Enhancement of Therapeutic Effectiveness (COMPETE III) randomized trial. Archives of Internal Medicine 2011;171(19):1736-44. - PubMed
Karlsson 2017 {published data only}
    1. Karlsson LO, Nilsson S, Charitakis E, Bång M, Johansson G, Nilsson L, et al. Clinical decision support for stroke prevention in atrial fibrillation (CDS-AF): rationale and design of a cluster randomized trial in the primary care setting. American Heart Journal 2017;187:45-52. - PubMed
Linder 2009 {published data only}
    1. Linder J, Schnipper J, Tsurikova R, Yu T, Volk L, Melnikas A, et al. Documentation-based clinical decision support to improve antibiotic prescribing for acute respiratory infections in primary care: a cluster randomised controlled trial. Journal of Innovation in Health Informatics 2009;17(4):231-40. - PubMed
Litvin 2013 {published data only}
    1. Litvin CB, Ornstein SM, Wessell AM, Nemeth LS, Nietert PJ. Use of an electronic health record clinical decision support tool to improve antibiotic prescribing for acute respiratory infections: the ABX-TRIP study. Journal of General Internal Medicine 2013;28(6):810-6. - PMC - PubMed
Mann 2014 {published data only}
    1. Mann D, Knaus M, McCullagh L, Sofianou A, Rosen L, McGinn T, et al. Measures of user experience in a streptococcal pharyngitis and pneumonia clinical decision support tools. Applied Clinical Informatics 2014;5(03):824-35. - PMC - PubMed
McGinn 2013 {published data only}
    1. McGinn TG, McCullagh L, Kannry J, Knaus M, Sofianou A, Wisnivesky JP, et al. Efficacy of an evidence-based clinical decision support in primary care practices: a randomized clinical trial. JAMA Internal Medicine 2013;173(17):1584-91. - PubMed
Melnick 2019 {published data only}
    1. Melnick ER, Jeffery MM, Dziura JD, Mao JA, Hess EP, Platts-Mills TF, et al. User-centred clinical decision support to implement emergency department-initiated buprenorphine for opioid use disorder: protocol for the pragmatic group randomised EMBED trial. BMJ Open 2019;9(5):e028488. - PMC - PubMed
Morgan 2011 {published data only}
    1. Morgan MB, Branstetter BF 4th, Clark C, House J, Baker D, Harnsberger HR. Just-in-time radiologist decision support: the importance of PACS-integrated workflow. Journal of the American College of Radiology 2011;8(7):497-500. - PubMed
Nam 2012 {published data only}
    1. Nam HS, Cha M‐J, Kim YD, Kim EH, Park E, Lee HS, et al. Use of a handheld, computerized device as a decision support tool for stroke classification. European Journal of Neurology 2012;19(3):426-30. - PubMed
Nendaz 2010 {published data only}
    1. Nendaz MR, Chopard P, Lovis C, Kucher N, Asmis LM, Dörffler J, et al. Adequacy of venous thromboprophylaxis in acutely ill medical patients (IMPART): multisite comparison of different clinical decision support systems. Journal of Thrombosis and Haemostasis 2010;8(6):1230-4. - PubMed
Peiris 2019 {published data only}
    1. Peiris D, Praveen D, Mogulluru K, Ameer MA, Raghu A, Li Q, et al. SMARThealth India: a stepped-wedge, cluster randomised controlled trial of a community health worker managed mobile health intervention for people assessed at high cardiovascular disease risk in rural India. PloS One 2019;14(3):e0213708. - PMC - PubMed
Prabhakaran 2019 {published data only}
    1. Prabhakaran D, Jha D, Prieto-Merino D, Roy A, Singh K, Ajay VS, et al. Effectiveness of an mHealth-based electronic decision support system for integrated management of chronic conditions in primary care: the mWellcare cluster-randomized controlled trial. Circulation 2019;139(3):380-91. - PubMed
Schaeffer 2019 {published data only}
    1. Schaeffer LE, Ahmed S, Rahman M, Whelan R, Rahman S, Roy AD, et al. Development and evaluation of a mobile application for case management of small and sick newborns in Bangladesh. BMC Medical Informatics and Decision Making 2019;19(1):116. - PMC - PubMed
Sturkenboom 2008 {published data only}
    1. Wyk JT, Wijk MA, Sturkenboom MC, Mosseveld M, Moorman PW, Lei J. Electronic alerts versus on-demand decision support to improve dyslipidemia treatment. Circulation 2008;117(3):371-8. - PubMed
Tajmir 2017 {published data only}
    1. Tajmir S, Raja AS, Ip IK, Andruchow J, Silveira P, Smith S, et al. Impact of clinical decision support on radiography for acute ankle injuries: a randomized trial. Western Journal of Emergency Medicine 2017;18(3):487. - PMC - PubMed
Tamblyn 2008 {published data only}
    1. Tamblyn R, Huang A, Taylor L, Kawasumi Y, Bartlett G, Grad R, et al. A randomized trial of the effectiveness of on-demand versus computer-triggered drug decision support in primary care. Journal of the American Medical Informatics Association 2008;15(4):430-8. - PMC - PubMed
Tebb 2019 {published data only}
    1. Tebb KP, Trieu SL, Rico R, Renteria R, Rodriguez F, Puffer M. A mobile health contraception decision support intervention for Latina adolescents: implementation evaluation for use in school-based health centers. JMIR mHealth and uHealth 2019;7(3):e11163. - PMC - PubMed
Wright 2012 {published data only}
    1. Wright A, Pang J, Feblowitz JC, Maloney FL, Wilcox AR, McLoughlin KS, et al. Improving completeness of electronic problem lists through clinical decision support: a randomized, controlled trial. Journal of the American Medical Informatics Association 2012;19(4):555-61. - PMC - PubMed

References to studies awaiting assessment

de Molina‐Férnandez 2019 {published data only}
    1. Molina-Férnandez MI, Raigal-Aran L, De la Flor-Lopez M, Prata P, Font-Jimenez I, Valls-Fonayet F, et al. The effectiveness of a digital shared decision-making tool in hormonal contraception during clinical assessment: study protocol of a randomized controlled trial in Spain. BMC Public Health 2019;19(1):1-8. - PMC - PubMed
Keitel 2017 {published data only}
    1. Keitel K, Kagoro F, Samaka J, Masimba J, Said Z, Temba H, et al. A novel electronic algorithm using host biomarker point-of-care tests for the management of febrile illnesses in Tanzanian children (e-POCT): a randomized, controlled non-inferiority trial. PLoS Medicine 2017;14(10):e1002411. [DOI: 10.1371/journal.pmed.1002411] [PMID: ] - DOI - PMC - PubMed
Khan 2020 {published data only}
    1. Khan AI, Mack JA, Salimuzzaman M, Zion MI, Sujon H, Ball RL, et al. Electronic decision support and diarrhoeal disease guideline adherence (mHDM): a cluster randomised controlled trial. Lancet Digital Health 2020;2(5):e250-8. [CLINICALTRIALS.GOV: NCT03154229] - PMC - PubMed

References to ongoing studies

NCT03311399 {published data only}
    1. NCT03311399. Using mHealth technology to identify and refer surgical site infections in Rwanda. clinicaltrials.gov/ct2/show/NCT03311399 (first received 17 October 2017). [CLINICALTRIALS.GOV: NCT03311399]

Additional references

Adepoju 2017
    1. Adepoju IO, Albersen BJ, De Brouwere V, Roosmalen J, Zweekhorst M. mHealth for clinical decision-making in Sub-Saharan Africa: a scoping review. JMIR mHealth and uHealth 2017;5(3):e38. - PMC - PubMed
Agarwal 2015
    1. Agarwal S, Perry HB, Long L-A, Labrique AB. Evidence on feasibility and effective use of mHealth strategies by frontline health workers in developing countries: systematic review. Tropical Medicine & International Health 2015;20(8):1003-14. - PMC - PubMed
Agarwal 2016
    1. Agarwal S, LeFevre AE, Lee J, L'Engle K, Mehl G, Sinha C, et al. Guidelines for reporting of health interventions using mobile phones: Mobile health (mHealth) evidence reporting and assessment (mERA) checklist. BMJ 2016;352:i1174. - PubMed
Arain 2010
    1. Arain M, Campbell MJ, Cooper CL, Lancaster GA. What is a pilot or feasibility study? A review of current practice and editorial policy. BMC Medical Research Methodology 2010;10(1):67. - PMC - PubMed
Awofeso 2004
    1. Awofeso N. What is the difference between 'primary care' and 'primary healthcare'? Quality in Primary Care 2004;12(2):93-4.
Baker 2007
    1. Baker B, Benton D, Friedman E, Russell A. Systems support for task-shifting to community health workers. chwcentral.org/sites/default/files/Systems%20Support%20for%20Task-Shifti... (accessed 2 October 2017).
Baptista 2018
    1. Baptista S, Sampaio ET, Heleno B, Azevedo LF, Martins C. Web-based versus usual care and other formats of decision aids to support prostate cancer screening decisions: systematic review and meta-analysis. Journal of Medical Internet Research 2018;20(6):e9070. - PMC - PubMed
Blank 2013
    1. Blank A, Prytherch H, Kaltschmidt H, Krings A, Sukums F, Mensah N, et al. "Quality of prenatal and maternal care: bridging the know-do gap" (QUALMAT study): an electronic clinical decision support system for rural Sub-Saharan Africa. BMC Medical Informatics and Decision Making 2013;13(1):44. - PMC - PubMed
Brenner 2016
    1. Brenner SK, Kaushal R, Grinspan Z, Joyce C, Kim I, Allard RJ, et al. Effects of health information technology on patient outcomes: a systematic review. Journal of the American Medical Informatics Association 2016;23(5):1016-36. - PMC - PubMed
Bright 2012
    1. Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, et al. Effect of clinical decision-support systems: a systematic review. Annals of Internal Medicine 2012;157(1):29-43. - PubMed
Carter 2019
    1. Carter J, Sandall J, Shennan AH, Tribe RM. Mobile phone apps for clinical decision support in pregnancy: a scoping review. BMC Medical Informatics and DEcision Making 2019;19:1. - PMC - PubMed
DistillerSR [Computer program]
    1. DistillerSR. Ottawa (Canada): Evidence Partners, (accessed 15 December 2017).
Divall 2013
    1. Divall P, Camosso-Stefinovic J, Baker R. The use of personal digital assistants in clinical decision making by health care professionals: a systematic review. Health Informatics Journal 2013;19(1):16-28. - PubMed
Dreesens 2019
    1. Dreesens D, Kremer L, Weijden T. The Dutch chaos case: a scoping review of knowledge and decision support tools available to clinicians in the Netherlands. Health Policy 2019;123(12):1288-97. - PubMed
Dussault 2006
    1. Dussault G, Franceschini MC. Not enough there, too many here: understanding geographical imbalances in the distribution of the health workforce. Human Resources for Health 2006;4(1):12. - PMC - PubMed
EPOC 2017a
    1. Cochrane Effective Practice and Organisation of Care (EPOC). Data collection form. EPOC resources for review authors, 2017. epoc.cochrane.org/resources/epoc-resources-review-authors (accessed prior to 9 June 2021).
EPOC 2017b
    1. Cochrane Effective Practice and Organisation of Care (EPOC). Suggested risk of bias criteria for EPOC reviews. EPOC resources for review authors, 2017. epoc.cochrane.org/resources/epoc-resources-review-authors (accessed prior to 9 June 2021).
EPOC 2017c
    1. Cochrane Effective Practice and Organisation of Care (EPOC). EPOC worksheets for preparing a 'Summary of findings' table using GRADE. EPOC resources for review authors, 2017. epoc.cochrane.org/resources/epoc-resources-review-authors (accessed prior to 9 June 2021).
Global Health Watch 2011
    1. Global Health Watch. Primary health care: a review and critical appraisal of its revitalization. www.ghwatch.org/sites/www.ghwatch.org/files/B1_0.pdf (accessed 2 October 2017).
GRADEpro GDT [Computer program]
    1. McMaster University (developed by Evidence Prime) GRADEpro GDT. Version (accessed 15 December 2017). Hamilton (ON): McMaster University (developed by Evidence Prime), 2015. Available at gradepro.org.
Guyatt 2008
    1. Guyatt GH, Oxman AD, Vist G, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al, GRADE Working Group. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008;336(7650):924-6. - PMC - PubMed
Higgins 2017
    1. Higgins JP, Savović J, Page MJ, Elbers RG, Sterne JA. Chapter 8: Assessing risk of bias in a randomized trial. In: Higgins JP, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al, editor(s). Cochrane Handbook for Systematic Reviews of Interventions version 5.2.0 (updated June 2017). Cochrane, 2017. Available from: handbook: training.cochrane.org/handbook/archive/v5.2.
Hunt 1998
    1. Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA 1998;280(15):1339:46. - PubMed
International Telecommunications Union 2015
    1. International Telecommunications Union. Information and Communication Technology (ICT) facts and figures. www.itu.int/en/ITU-D/Statistics/Documents/facts/ICTFactsFigures2015.pdf (accessed 2 October 2017).
Jaspers 2011
    1. Jaspers MW, Smeulers M, Vermeulen H, Peute LW. Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings. Journal of the American Medical Informatics Association 2011;18(3):327-34. - PMC - PubMed
Kaplan 2001
    1. Kaplan B. Evaluating informatics applications – clinical decision support systems literature review. International Journal of Medical Informatics 2001;64(1):15-37. - PubMed
Kawamoto 2005
    1. Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 2005;330(7494):765. - PMC - PubMed
Kilsdonk 2017
    1. Kilsdonk E, Peute LW, Jaspers MW. Factors influencing implementation success of guideline-based clinical decision support systems: a systematic review and gaps analysis. International Journal of Medical Informatics 2017;1(98):56-64. - PubMed
Kocher 2010
    1. Kocher R, Emanuel EJ, DeParle NA. The Affordable Care Act and the future of clinical medicine: the opportunities and challenges. Annals of Internal Medicine 2010;153(8):536-9. - PubMed
Kruk 2018
    1. Kruk ME, Gage AD, Arsenault C, Jordan K, Leslie HH, Roder-DeWan S, et al. High-quality health systems in the Sustainable Development Goals era: time for a revolution. Lancet Global Health Commission 2018;6:e1196. [DOI: ] - PMC - PubMed
Lefebvre 2011
    1. Lefebvre C, Manheimer E, Glanville J. Chapter 6: Searching for studies. In: Higgins JP, Green S, editor(s). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available at: handbook: training.cochrane.org/handbook/archive/v5.1/.
Lehmann 2008
    1. Lehmann U, Dieleman M, Martineau T. Staffing remote rural areas in middle- and low-income countries: a literature review of attraction and retention. BMC Health Services Research 2008;8(1):19. - PMC - PubMed
Liberati 2009
    1. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Medicine 2009;6(7):e1000100. - PMC - PubMed
Martínez‐Pérez 2014
    1. Martínez-Pérez B, la Torre-Díez I, López-Coronado M, Sainz-de-Abajo B, Robles M, García-Gómez JM. Mobile clinical decision support systems and applications: a literature and commercial review. Journal of Medical Systems 2014;38(1):4. - PubMed
Mazo 2020
    1. Mazo C, Kearns C, Mooney C, Gallagher WM. Clinical decision support systems in breast cancer: a systematic review. Cancers 2020;12(2):369. - PMC - PubMed
McKenzie 2021
    1. McKenzie JE, Brennan SE. Synthesizing and presenting findings using other methods. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al, editors(s). Cochrane Handbook for Systematic Reviews of Interventions version 6.2 . Cochrane, February 2021.
Mickan 2014
    1. Mickan S, Atherton H, Roberts NW, Heneghan C, Tilson JK. Use of handheld computers in clinical practice: a systematic review. BMC Medical Informatics and Decision Making 2014;14(1):56. - PMC - PubMed
Middleton 2016
    1. Middleton B, Sittig DF, Wright A. Clinical decision support: a 25 year retrospective and a 25 year vision. Yearbook of Medical Informatics 2016;25(S01):S103-16. - PMC - PubMed
Mishra 2019
    1. Mishra SR, Lygidakis C, Neupane D, Gyawali B, Uwizihiwe JP, Virani SS, et al. Combating non-communicable diseases: potentials and challenges for community health workers in a digital age: a narrative review of the literature. Health Policy and Planning 2019;34(1):55-66. - PubMed
Mohanan 2015
    1. Mohanan M, Vera-Hernández M, Das V, Giardili S, Goldhaber-Fiebert JD, Rabin TL, et al. The know-do gap in quality of health care for childhood diarrhea and pneumonia in rural India. JAMA Pediatrics 2015;169(4):349-57. - PMC - PubMed
Moja 2014
    1. Moja L, Kwag KH, Lytras T, Bertizzolo L, Pecoraro V, Rigon G, et al. Effectiveness of computerized decision support systems linked to electronic health records: a systematic review and meta-analysis. American Journal of Public Health 2014;104(12):e12-22. - PMC - PubMed
Muldoon 2006
    1. Muldoon LK, Hogg WE, Levitt M. Primary care (PC) and primary health care (PHC): what is the difference? Canadian Journal of Public Health/Revue Canadienne de Santé Publique 2006;409(5):409-11. - PMC - PubMed
Njie 2015
    1. Njie GJ, Proia KK, Thota AB, Finnie RK, Hopkins DP, Banks SM, et al. Clinical decision support systems and prevention: a community guide cardiovascular disease systematic review. American Journal of Preventive Medicine 2015;49(5):784-95. - PMC - PubMed
Odendaal 2020
    1. Odendaal WA, Watkins JA, Leon N, Goudge J, Griffiths F, Tomlinson M, et al. Health workers' perceptions and experiences of using mHealth technologies to deliver primary healthcare services: a qualitative evidence synthesis. Cochrane Database of Systematic Reviews 2020, Issue 1. Art. No: CD011942. [DOI: 10.1002/14651858.CD011942.pub2] - DOI - PMC - PubMed
Orton 2018
    1. Orton M, Agarwal S, Muhoza P, Vasudevan L, Vu A. Strengthening delivery of health services using digital devices. Global Health: Science and Practice 2018;6(1):S61-71. - PMC - PubMed
Rednick 2014
    1. Rednick C, Dini HS, Long LA. The current state of CHW training programs in Sub-Saharan Africa and South Asia: what we know, what we don't know, and what we need to do. 1millionhealthworkers.org/files/2013/01/1mCHW_mPowering_LitReview_Format... (accessed 2 October 2017).
Review Manager 2014 [Computer program]
    1. Nordic Cochrane Centre, The Cochrane Collaboration Review Manager 5 (RevMan 5). Version 5.3. Copenhagen: Nordic Cochrane Centre, The Cochrane Collaboration, 2014.
Richardson 2019
    1. Richardson KM, Fouquet SD, Kerns E, McCulloh RJ. Impact of mobile device-based clinical decision support tool on guideline adherence and mental workload. Academic Pediatrics 2019;19(7):828-34. - PMC - PubMed
Schünemann 2017
    1. Schünemann HJ, Oxman AD, Higgins JP, Vist GE, Glasziou P, Akl E, et al, on behalf of the Cochrane GRADEing Methods Group and the Cochrane Statistical Methods Group. Chapter 11: Completing 'Summary of findings' tables and grading the confidence in or quality of the evidence. In: Higgins JP, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al, editor(s). Cochrane Handbook for Systematic Reviews of Interventions version 5.2.0 (updated June 2017). Cochrane, 2017. Available from: handbook: training.cochrane.org/handbook/archive/v5.2.
Sterne 2011
    1. Sterne JA, Sutton AJ, Ioannidis JP, Terrin N, Jones DR, Lau J, et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ 2011;343:d4002. [DOI: 10.1136/bmj.d4002] - DOI - PubMed
Sutton 2020
    1. Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, strategies for success. npj Digital Medicine 2020;3:17. - PMC - PubMed
Syrowatka 2016
    1. Syrowatka A, Krömker D, Meguerditchian AN, Tamblyn R. Features of computer-based decision aids: systematic review, thematic synthesis, and meta-analyses. Journal of Medical Internet Research 2016;18(1):e20. - PMC - PubMed
Tong 2021
    1. Tong G, Geng Q, Wang D, Liu T. Web-based decision aids for cancer clinical decisions: a systematic review and meta-analysis. Support Care Cancer 2021 Apr 8 [Epub ahead of print]. [DOI: ] - PubMed
Varghese 2018
    1. Varghese J, Kleine M, Gessner SI, Sandmann S, Dugas M. Effects of computerized decision support system implementations on patient outcomes in inpatient care: a systematic review. Journal of the American Medical Informatics Association 2018;25(5):593-602. - PMC - PubMed
Wallace 2017
    1. Wallace BC, Noel-Storr A, Marshall IJ, Cohen AM, Smalheiser NR, Thomas J. Identifying reports of randomized controlled trials (RCTs) via a hybrid machine learning and crowdsourcing approach. Journal of the American Medical Informatics Association 2017;24(6):1165-8. - PMC - PubMed
WHO 2016
    1. World Health Organization. Health workforce requirements for universal health coverage and sustainable development goals. apps.who.int/iris/bitstream/handle/10665/250330/9789241511407-eng.pdf (accessed 25 March 2021).
WHO 2019
    1. World Health Organization. WHO guideline: recommendations on digital interventions for health systems strengthening, 2019. www.who.int/reproductivehealth/publications/digital-interventions-health... (accessed 9 June 2021). - PubMed
WHO 2020
    1. World Health Organization. Global strategy on human resources for health: Workforce 2030. www.who.int/hrh/resources/global_strategy_workforce2030_14_print.pdf (accessed 25 March 2021).
Yau 2019
    1. Yau M, Timmerman V, Zwarenstein M, Mayers P, Cornick RV, Bateman E, et al. e-PC101: an electronic clinical decision support tool developed in South Africa for primary care in low-income and middle-income countries. BMJ Global Health 2019;3:e001093. - PMC - PubMed

References to other published versions of this review

Agarwal 2018
    1. Agarwal S, Tamrat T, Glenton C, Lewin S, Henschke N, Maayan N, et al. Decision‐support tools via mobile devices to improve quality of care in primary healthcare settings. Cochrane Database of Systematic Reviews 2018, Issue 2. Art. No: CD012944. [DOI: 10.1002/14651858.CD012944] - DOI - PMC - PubMed

Publication types

LinkOut - more resources