Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Aug 13;8(8):CD012012.
doi: 10.1002/14651858.CD012012.pub2.

Routine Health Information System (RHIS) improvements for strengthened health system management

Natalie Leon  1   2 , Yusentha Balakrishna  3 , Ameer Hohlfeld  4 , Willem A Odendaal  1   5 , Bey-Marrié Schmidt  4 , Virginia Zweigenthal  6   7 , Jocelyn Anstey Watkins  8 , Karen Daniels  1   7
Affiliations

Routine Health Information System (RHIS) improvements for strengthened health system management

Natalie Leon et al. Cochrane Database Syst Rev. .

Abstract

Background: A well-functioning routine health information system (RHIS) can provide the information needed for health system management, for governance, accountability, planning, policy making, surveillance and quality improvement, but poor information support has been identified as a major obstacle for improving health system management.

Objectives: To assess the effects of interventions to improve routine health information systems in terms of RHIS performance, and also, in terms of improved health system management performance, and improved patient and population health outcomes.

Search methods: We searched the Cochrane Central Register of Controlled Trials (CENTRAL) in the Cochrane Library, MEDLINE Ovid and Embase Ovid in May 2019. We searched Global Health, Ovid and PsycInfo in April 2016. In January 2020 we searched for grey literature in the Grey Literature Report and in OpenGrey, and for ongoing trials using the International Clinical Trials Registry Platform (ICTRP) and ClinicalTrials.gov. In October 2019 we also did a cited reference search using Web of Science, and a 'similar articles' search in PubMed.

Selection criteria: Randomised and non-randomised trials, controlled before-after studies and time-series studies comparing routine health information system interventions, with controls, in primary, hospital or community health care settings. Participants included clinical staff and management, district management and community health workers using routine information systems.

Data collection and analysis: Two authors independently reviewed records to identify studies for inclusion, extracted data from the included studies and assessed the risk of bias. Interventions and outcomes were too varied across studies to allow for pooled risk analysis. We present a 'Summary of findings' table for each intervention comparisons broadly categorised into Technical and Organisational (or a combination), and report outcomes on data quality and service quality. We used the GRADE approach to assess the certainty of the evidence.

Main results: We included six studies: four cluster randomised trials and two controlled before-after studies, from Africa and South America. Three studies evaluated technical interventions, one study evaluated an organisational intervention, and two studies evaluated a combination of technical and organisational interventions. Four studies reported on data quality and six studies reported on service quality. In terms of data quality, a web-based electronic TB laboratory information system probably reduces the length of time to reporting of TB test results, and probably reduces the overall rate of recording errors of TB test results, compared to a paper-based system (moderate certainty evidence). We are uncertain about the effect of the electronic laboratory information system on the recording rate of serious (misidentification) errors for TB test results compared to a paper-based system (very low certainty evidence). Misidentification errors are inaccuracies in transferring test results between an electronic register and patients' clinical charts. We are also uncertain about the effect of the intervention on service quality (timeliness of starting or changing a patient's TB treatment) (very low certainty evidence). A hand-held electronic device probably improves the length of time to report TB test results, and probably reduces the total frequency of recording errors in TB test results between the laboratory notebook and the electronic information record system, compared to a paper-based system (moderate-certainty evidence). We are, however, uncertain about the effect of the intervention on the frequency of serious (misidentification) errors in recording between the laboratory notebook and the electronic information record, compared to a paper-based system (very low certainty evidence). We are uncertain about the effect of a hospital electronic health information system on service quality (length of time outpatients spend at hospital, length of hospital stay, and hospital revenue collection), compared to a paper-based system (very low certainty evidence). High-intensity brief text messaging (SMS) may make little or no difference to data quality (in terms of completeness of documentation of pregnancy outcomes), compared to low-intensity brief text messaging (low-certainty evidence). We are uncertain about the effect of electronic drug stock notification (with either data management support or product transfer support) on service quality (in terms of transporting stock and stock levels), compared to paper-based stock notification (very low certainty evidence). We are uncertain about the effect of health information strengthening (where it is part of comprehensive service quality improvement intervention) on service quality (health worker motivation, receipt of training by health workers, health information index scores, quality of clinical observation of children and adults) (very low certainty evidence).

Authors' conclusions: The review indicates mixed effects of mainly technical interventions to improve data quality, with gaps in evidence on interventions aimed at enhancing data-informed health system management. There is a gap in interventions studying information support beyond clinical management, such as for human resources, finances, drug supply and governance. We need to have a better understanding of the causal mechanisms by which information support may affect change in management decision-making, to inform robust intervention design and evaluation methods.

PubMed Disclaimer

Conflict of interest statement

Natalie Leon: no known conflicts of interest.

Yusentha Balakrishna: no known conflicts of interest.

Ameer Hohlfield: no known conflicts of interest.

Willem Odendaal: no known conflicts of interest.

Bey‐Marrié Schmidt: no known conflicts of interest.

Virginia Zweigenthal: no known conflicts of interest,

Jocelyn Anstey Watkins: no known conflict of interest.

Karen Daniels: no known conflicts of interest.

Figures

1
1
Components of RHISs (adapted from the PRISM Framework)
2
2
PRISMA Flow diagram for RHIS systematic review
3
3
Risk of bias summary: review authors' judgements about each risk of bias item for each included study
4
4
Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies
1.1
1.1. Analysis
Comparison 1: Web‐based electronic TB laboratory information system compared to paper‐based system (Blaya 2014), Outcome 1: Length of time to report TB culture test results
1.2
1.2. Analysis
Comparison 1: Web‐based electronic TB laboratory information system compared to paper‐based system (Blaya 2014), Outcome 2: Length of time to report TB drug susceptibility test results
1.3
1.3. Analysis
Comparison 1: Web‐based electronic TB laboratory information system compared to paper‐based system (Blaya 2014), Outcome 3: Recording errors of TB culture test results (Overall)
1.4
1.4. Analysis
Comparison 1: Web‐based electronic TB laboratory information system compared to paper‐based system (Blaya 2014), Outcome 4: Recording errors of TB drug susceptibility test results (Overall)
1.5
1.5. Analysis
Comparison 1: Web‐based electronic TB laboratory information system compared to paper‐based system (Blaya 2014), Outcome 5: Recording errors: misidentification errors for TB culture test results
1.6
1.6. Analysis
Comparison 1: Web‐based electronic TB laboratory information system compared to paper‐based system (Blaya 2014), Outcome 6: Recording errors: misidentification errors for TB drug susceptibility test results
1.7
1.7. Analysis
Comparison 1: Web‐based electronic TB laboratory information system compared to paper‐based system (Blaya 2014), Outcome 7: Timeliness of starting or changing a patient's TB treatment
2.1
2.1. Analysis
Comparison 2: Hand‐held electronic device for collecting TB laboratory information compared to a paper‐based system (Blaya 2009), Outcome 1: Length of time to report TB culture test results
2.2
2.2. Analysis
Comparison 2: Hand‐held electronic device for collecting TB laboratory information compared to a paper‐based system (Blaya 2009), Outcome 2: Length of time to report TB smear test results
2.3
2.3. Analysis
Comparison 2: Hand‐held electronic device for collecting TB laboratory information compared to a paper‐based system (Blaya 2009), Outcome 3: Recording errors
2.4
2.4. Analysis
Comparison 2: Hand‐held electronic device for collecting TB laboratory information compared to a paper‐based system (Blaya 2009), Outcome 4: Recording errors: misidentification errors
4.1
4.1. Analysis
Comparison 4: Brief text messaging (SMS) compared to low‐intensity brief text messaging for community based surveillance of pregnancy outcomes (Joos 2016), Outcome 1: Documentation of matched pregnancy outcome data
7.1
7.1. Analysis
Comparison 7: Health information strengthening as part of comprehensive quality improvement compared to no quality improvement (Mutale 2014), Outcome 1: Health worker motivation
7.2
7.2. Analysis
Comparison 7: Health information strengthening as part of comprehensive quality improvement compared to no quality improvement (Mutale 2014), Outcome 2: Health worker training
7.3
7.3. Analysis
Comparison 7: Health information strengthening as part of comprehensive quality improvement compared to no quality improvement (Mutale 2014), Outcome 3: Health information index
7.4
7.4. Analysis
Comparison 7: Health information strengthening as part of comprehensive quality improvement compared to no quality improvement (Mutale 2014), Outcome 4: Clinical observation index ‐ children
7.5
7.5. Analysis
Comparison 7: Health information strengthening as part of comprehensive quality improvement compared to no quality improvement (Mutale 2014), Outcome 5: Clinical observation index ‐ adults
See this image and copyright information in PMC

Update of

  • doi: 10.1002/14651858.CD012012

References

References to studies included in this review

Blaya 2009 {published data only}
    1. Blaya JA, Cohen T, Rodriguez P, Kim J, Fraser HS. Personal digital assistants to collect tuberculosis bacteriology data in Peru reduce delays, errors, and workload, and are acceptable to users: cluster randomized controlled trial. International Journal of Infectious Diseases 2009;13(3):410-8. [DOI: ] [PMID: ] - PMC - PubMed
Blaya 2014 {published data only}
    1. Blaya J, Shin S, Yagui M, Contreras C, Cegielski P, Yale G, et al. Reducing communication delays and improving quality of care with a tuberculosis laboratory information system in resource poor environments: a cluster randomized controlled trial. PloS One 2014;9(4):e90110. [DOI: ] - PMC - PubMed
    1. Blaya J, Shin S, Yale G, Suarez C, Asencios L, Contreras C, et al. Electronic laboratory system reduces errors in National Tuberculosis Program: a cluster randomized controlled trial. International Journal of Tuberculosis and Lung Disease 2010;2214(8):1009-15. - PMC - PubMed
    1. Blaya J, Shin S, Yale G, Suarez C, Asencios L, Contreras C, et al. Full impact of laboratory information system requires direct use by clinical staff: cluster randomized controlled trial. Journal of the American Medical Informatics Association 2011;18(1):11-6. [DOI: 10.1136/jamia.2010.005280] - DOI - PMC - PubMed
Joos 2016 {published data only}
    1. Joos O, Silva R, Amouzou A, Moulton LH, Perin J, Bryce J, et al. Evaluation of a mhealth data quality intervention to improve documentation of pregnancy outcomes by health surveillance assistants in Malawi: a cluster randomized trial. PloS One 2016;11(1):e0145238. [DOI: ] [PMID: ] - PMC - PubMed
Mbananga 2002 {unpublished data only}
    1. Mbananga N, Madale R, Becker P. Evaluation of hospital information system in the Northern Province in South Africa: “Using Outcome Measures”. Available at pdfs.semanticscholar.org/ab97/8903cb3089607cedc1c72343e9d144938807.pdf.
Mutale 2014 {published data only (unpublished sought but not used)}
    1. Mutale W, Stringer J, Chintu N, Chilengi R, Mwanamwenge MT, Kasese N, et al. Application of balanced scorecard in the evaluation of a complex health system intervention: 12 months post intervention findings from the BHOMA intervention: a cluster randomised trial in Zambia. PLOS One 2014;9(4):e93977. [DOI: 10.1371/journal.pone.0093977] - DOI - PMC - PubMed
SC4CCM 2013 {published and unpublished data}
    1. SC4CCM. Malawi Community Health Supply Chain Midline Assessment Report 2013. 1rqxbs47ujl4rdy6q3nzf554.wpengine.netdna-cdn.com/wp-content/uploads/2016....
    1. Shieshia M, Noel M, Andersson S, Felling B, Alva S, Agarwal S, et al. Strengthening community health supply chain performance through an integrated approach: using mHealth technology and multilevel teams in Malawi. Journal of Global Health 2014;4(2):020406. [DOI: 10.7189/jogh.04.020406] - DOI - PMC - PubMed

References to studies excluded from this review

Agrawal 2009 {published data only}
    1. Agrawal A, Wu W. Reducing medication errors and improving systems reliability using an electronic medication reconciliation system. Joint Commission Journal on Quality and Patient Safety 2009;35(2):106-14. [DOI: ] - PubMed
Ammenwerth 2001 {published data only}
    1. Ammenwerth E, Eichstadter R, Haux R, Pohl U, Rebel S, Ziegler S. A randomized evaluation of a computer-based nursing documentation system. Methods of Information in Medicine 2001;40(2):61-8. [DOI: 10.1055/s-0038-1634465] - DOI - PubMed
Andersson 2013 {published data only}
    1. Andersson ML, Bottiger Y, Lindh JD, Wettermark B, Eiermann B. Impact of the drug-drug interaction database SFINX on prevalence of potentially serious drug-drug interactions in primary health care. European Journal of Clinical Pharmacology 2013;69(3):565-71. [DOI: ] - PubMed
Boockvar 2017 {published data only}
    1. Boockvar KS, Ho W, Pruskowski J, DiPalo KE, Wong JJ, Patel J, et al. Effect of health information exchange on recognition of medication discrepancies is interrupted when data charges are introduced: results of a cluster-randomized controlled trial. Journal of the American Medical Informatics Association 2017;24(6):1095-101. [DOI: ] - PMC - PubMed
Brugha 1996 {published data only}
    1. Brugha RF, Kevany JP. Maximizing immunization coverage through home visits: a controlled trial in an urban area of Ghana. Bulletin of the World Health Organization 1996;74(5):517-24. - PMC - PubMed
Cawsey 2000 {published data only}
    1. Cawsey AJ, Jones RB, Pearson J. The evaluation of a personalised health information system for patients with cancer. User Modeling and User-adapted Interaction 2000;10(1):47-72. [DOI: ]
Chang 2011 {published data only}
    1. Chang LW, Kagaayi J, Arem H, Nakigozi G, Ssempijja V, Serwadda D, et al. Impact of a mHealth intervention for peer health workers on AIDS care in rural Uganda: a mixed methods evaluation of a cluster-randomized trial. AIDS and Behavior 2011;15(8):1776-84. [DOI: ] - PMC - PubMed
Chen 2007 {published data only}
    1. Chen T-H, Li L, Sigle JM, Du Y-P, Wang H-M, Lei J. Crossover randomized controlled trial of the electronic version of the Chinese SF-36. Journal of Zhejiang University Science B 2007;8(8):604-8. [DOI: ] - PMC - PubMed
Choi 2004 {published data only}
    1. Choi SS, Jazayeri DG, Mitnick CD, Chalco K, Bayona J, Fraser HS. Implementation and initial evaluation of a web-based nurse order entry system for multidrug-resistant tuberculosis patients in Peru. Studies in Health Technology and Informatics 2004;107(Pt1):202-6. [DOI: 10.3233/978-1-60750-949-3-202] - DOI - PubMed
Chrischilles 2014 {published data only}
    1. Chrischilles EA, Hourcade JP, Doucette W, Eichmann D, Gryzlak B, Lorentzen R, et al. Personal health records: a randomized trial of effects on elder medication safety. Journal of the American Medical Informatics Association 2014;21(4):679-86. [DOI: ] - PMC - PubMed
de Lusignan 2004 {published data only}
    1. Lusignan S, Hague N, Brown A, Majeed A. An educational intervention to improve data recording in the management of ischaemic heart disease in primary care. Journal of Public Health 2004;26(1):34-7. [DOI: ] - PubMed
Dixon 2017 {published data only}
    1. Dixon BE, Barboza K, Jensen AE, Bennett KJ, Sherman SE, Schwartz MD. Measuring practicing clinicians' information literacy. An exploratory analysis in the context of panel management. Applied Clinical Informatics 2017;8(1):149-61. [DOI: 10.4338/ACI-2016-06-RA-0083.] - PMC - PubMed
Dowding 2012 {published data only}
    1. Dowding DW, Turley M, Garrido T. The impact of an electronic health record on nurse sensitive patient outcomes: an interrupted time series analysis. Journal of the American Medical Informatics Association 2012;19(4):615-20. [DOI: ] - PMC - PubMed
Dreischulte 2016 {published data only}
    1. Dreischulte T, Donnan P, Grant A, Hapca A, McCowan C, Guthrie B. Safer Prescribing — A Trial of Education, Informatics, and Financial Incentives. The New England Journal of Medicine 2016;374(11):1053-64. - PubMed
Ekwueme 2008 {published data only}
    1. Ekwueme OC, Aghaji MN. Can retraining enhance the knowledge and attitude of primary health workers towards health management information system in Enugu State, Nigeria? International Journal of Medicine and Health Development 2008;13(2):71-7.
Escobar‐Perez 2016 {published data only}
    1. Escobar-Perez B, Escobar-Rodriguez T, Bartual-Sopena L. Integration of healthcare and financial information: evaluation in a public hospital using a comprehensive approach. Health Informatics Journal 2016;22(4):878-96. [DOI: 10.1177/1460458215595259] - DOI - PubMed
Eurlings 1997 {published data only}
    1. Eurlings F, Asten A, Cozijn H, Klaassen K, Stokman R, Valkenburg R, et al. Effects of a nursing information system in 5 Dutch hospitals. Studies in Health Technology and Informatics 1997;46:50-5. [DOI: 10.3233/978-1-60750-890-8-50] - DOI - PubMed
Fidahussein 2011 {unpublished data only}
    1. Fidahussein M, Hook J, Kesterson J, Were M. Using a regional health information exchange to improve identification of post-discharge follow-up providers. In: Society of General Internal Medicine, 34th Annual Meeting, Phoenix, Arizona, May 4–7, 2011. 2011:S163-4.
Field 2009 {published data only}
    1. Field TS, Rochon P, Lee M, Gavendo L, Baril JL, Gurwitz JH. Computerized clinical decision support during medication ordering for long-term care residents with renal insufficiency. Journal of the American Medical Informatics Association : JAMIA 2009;16(4):480-5. [DOI: ] - PMC - PubMed
Filler 2017 {published data only}
    1. Filler S. Strengthening routine health information systems to target malaria control implementation and optimize evaluation of impact. American Journal of Tropical Medicine and Hygiene 2017;97(3 Suppl):2. [DOI: 10.4269/ajtmh.17-0581] - DOI - PMC - PubMed
Frame 1994 {published data only}
    1. Frame PS, Zimmer JG, Werth PL, Hall WJ, Eberly SW. Computer-based vs manual health maintenance tracking. A controlled trial. Archives of Family Medicine 1994;3(7):581-8. [DOI: ] - PubMed
Freundlich 2013 {published data only}
    1. Freundlich RE, Barnet CS, Mathis MR, Shanks AM, Tremper KK, Kheterpal S. A randomized trial of automated electronic alerts demonstrating improved reimbursable anesthesia time documentation. Journal of Clinical Anesthesia 2013;25(2):110-4. [DOI: 10.1016/j.jclinane.2012.06.020] - DOI - PubMed
Gernant 2018 {published data only}
    1. Gernant SA, Zillich AJ, Snyder ME. Access to medical records' impact on community pharmacist-delivered medication therapy management: a pilot from the Medication Safety Research Network of Indiana (Rx-SafeNet). Journal of Pharmacy Practice 2018;31(6):642-50. [DOI: 10.1177/0897190017735422] - DOI - PubMed
Gisore 2012 {published data only}
    1. Gisore P, Shipala E, Otieno K, Rono B, Marete I, Tenge C et al. Community based weighing of newborns and use of mobile phones by village elders in rural settings in Kenya: a decentralised approach to health care provision. BMC Pregnancy and Childbirth 2012;12(15):1-10. [DOI: 10.1186/1471-2393-12-15] - DOI - PMC - PubMed
Gong 2016 {published data only}
    1. Gong Y, Hua L, Wang S. Leveraging user's performance in reporting patient safety events by utilizing text prediction in narrative data entry. Computer Methods and Programs in Biomedicine 2016;131:181-9. [DOI: 10.1016/j.cmpb.2016.03.031] - DOI - PMC - PubMed
Grandville 2006 {published data only}
    1. Grandville T, Molinari A, Campbell S, Dwyer C, Scocozza K. Effect of CPOE on medication errors. American Journal of Health-system Pharmacy 2006;63(5):409. [DOI: ] - PubMed
Grischott 2018 {published data only}
    1. Grischott T, Zechmann S, Rachamin Y, Markun S, Chmiel C, Senn O, et al. Improving inappropriate medication and information transfer at hospital discharge: study protocol for a cluster RCT. Implementation Science 2018;13(1):155. [DOI: ] - PMC - PubMed
Guiriguet‐Capdevila 2014 {published data only}
    1. Guiriguet-Capdevila C, Munoz-Ortiz L, Rivero-Franco I, Vela-Vallespin C, Vilarrubi-Estrella M, Torres-Salinas M, et al. Can an alert in primary care electronic medical records increase participation in a population-based screening programme for colorectal cancer? COLO-ALERT, a randomised clinical trial. BMC Cancer 2014;14:232. [DOI: ] - PMC - PubMed
Gustafson 1999 {published data only}
    1. Gustafson DH, Hawkins R, Boberg E, Pingree S, Serlin RE, Graziano F, et al. Impact of a patient-centered, computer-based health information/support system. American Journal of Preventive Medicine 1999;16(1):1-9. [DOI: ] - PubMed
Hammond 1990 {published data only}
    1. Hammond KW, Prather RJ, Date VV, King CA. A provider-interactive medical record system can favorably influence costs and quality of medical care. Computers in Biology and Medicine 1990;20(4):267-79. [DOI: ] - PubMed
Haskew 2015 {published data only}
    1. Haskew J, Ro G, Saito K, Turner K, Odhiambo G, Wamae A, et al. Implementation of a cloud-based electronic medical record for maternal and child health in rural Kenya. International Journal of Medical Informatics 2015;84(5):349-54. [DOI: ] - PubMed
Hassink 2013 {published data only}
    1. Hassink JJ, Essenberg MD, Roukema JA, den Bemt PM. Effect of bar-code-assisted medication administration on medication administration errors. American Journal of Health-system Pharmacy 2013;70(7):572-3. [DOI: ] - PubMed
Hebel 2012 {published data only}
    1. Hebel E, Middleton B, Shubina M, Turchin A. Bridging the chasm: effect of health information exchange on volume of laboratory testing. Archives of Internal Medicine 2012;172(6):517-9. [DOI: 10.1001/archinternmed.2011.2104] - DOI - PubMed
Heidarizadeh 2017 {published data only}
    1. Heidarizadeh K, Rassouli M, Manoochehri H, Tafreshi MZ, Ghorbanpour RK. Effect of electronic report writing on the quality of nursing report recording. Electronic Physician 2017;9(10):5439-45. [DOI: 10.19082/5439] - DOI - PMC - PubMed
Hendriks 2016 {published data only}
    1. Hendriks PH, Ligthart PE, Schouteten RL. Knowledge management, health information technology and nurses' work engagement. Health Care Management Review 2016;41(3):256-66. [DOI: 10.1097/HMR.0000000000000075] - DOI - PubMed
Hooper 2012 {published data only}
    1. Hooper MH, Weavind L, Wheeler AP, Martin JB, Gowda SS, Semler MW, et al. Randomized trial of automated, electronic monitoring to facilitate early detection of sepsis in the intensive care unit. Critical Care Medicine 2012;40(7):2096-101. [DOI: 10.1097/CCM.0b013e318250a887] - DOI - PMC - PubMed
Hunt 2009 {published data only}
    1. Hunt JS, Siemienczuk J, Gillanders W, LeBlanc BH, Rozenfeld Y, Bonin K, et al. The impact of a physician-directed health information technology system on diabetes outcomes in primary care: a pre- and post-implementation study. Informatics in Primary Care 2009;17(3):165-74. [DOI: ] - PubMed
Ir 2015 {published data only}
    1. Ir P, Korachais C, Chheng K, Horemans D, Damme W, Meessen B. Boosting facility deliveries with results-based financing: a mixed-methods evaluation of the government midwifery incentive scheme in Cambodia. BMC Pregnancy and Childbirth 2015;15:170. [DOI: ] - PMC - PubMed
Iyer 2015 {published data only}
    1. Iyer H, Hirschhorn L, Kamanzi E, Drobac P, Cyamatare F, Nahimana E, et al. Impact of a district-wide health center strengthening intervention on healthcare utilization in rural Rwanda: An interrupted time series analysis. Annals of Global Health 2015;81(1):181. [DOI: 10.1016/j.aogh.2015.02.915] - DOI
Iyer 2017 {published data only}
    1. Iyer HS, Hirschhorn LR, Nisingizwe MP, Kamanzi E, Drobac PC, Rwabukwisi FC, et al. Impact of a district-wide health center strengthening intervention on healthcare utilization in rural Rwanda: Use of interrupted time series analysis. PloS One 2017;12(8):e0182418. [DOI: ] - PMC - PubMed
Ji 2018 {published data only}
    1. Ji C, Quinn T, Gavalova L, Lall R, Scomparin C, Horton J, et al. Feasibility of data linkage in the PARAMEDIC trial: a cluster randomised trial of mechanical chest compression in out-of-hospital cardiac arrest. BMJ Open 2018;8(7):e021519. [DOI: ] - PMC - PubMed
Johnson 2016 {published data only}
    1. Johnson M, Sanchez P, Zheng C. Reducing patient clinical management errors using structured content and electronic nursing handover. Journal of Nursing Care Quality 2016;31(3):245-53. [DOI: 10.1097/NCQ.0000000000000167] - DOI - PubMed
Lamanna, 2019 {published data only}
    1. Lamanna C, Byrne L. A pilot study of a novel, incentivised mHealth technology to monitor the vaccine supply chain in rural Zambia. Pan African Medical Journal 2019;33(50):1-8. [DOI: 10.11604/pamj.2019.33.50.16318] - DOI - PMC - PubMed
Lester 2006 {published data only}
    1. Lester WT, Grant RW, Barnett GO, Chueh HC. Randomized controlled trial of an informatics-based intervention to increase statin prescription for secondary prevention of coronary disease. Journal of General Internal Medicine 2006;21(1):22-9. [DOI: ] - PMC - PubMed
Lester 2010 {published data only}
    1. Lester RT, Ritvo P, Mills EJ, Kariri A, Karanja S, Chung MH, et al. Effects of a mobile phone short message service on antiretroviral treatment adherence in Kenya (WelTel Kenya1): a randomised trial. Lancet 2010;376(9755):1838-45. [DOI: 10.1016/S0140-6736(10)61997-6] - DOI - PubMed
Lin 2010 {published data only}
    1. Lin IC, Hou YH, Huang HL, Chu TB, Chang TE. Managing nursing assistants with a web-based system: an empirical investigation of the mixed-staff strategy. Journal of Medical Systems 2010;34(3):341-8. [DOI: ] - PubMed
Pop‐Eleches 2011 {published data only}
    1. Pop-Eleches C, Tirumurthy H, Habyarimana JP, Zivin JG, Goldstein MP, Walque D, et al. Mobile phone technologies improve adherence to antiretroviral treatment in a resource-limited setting: a randomized controlled trial of text message reminders. AIDS (London, England) 2011;25(6):825-34. [DOI: 10.1097/QAD.0b013e32834380c1] - DOI - PMC - PubMed
Rauhala 2008 {unpublished data only}
    1. Rauhala, A. The Validity and Feasibility of Measurement Tools for Human Resources Management in Nursing Case of the RAFAELA System. Available at epublications.uef.fi/pub/urn_isbn_978-951-27-1069-0/urn_isbn_978-951-27-....
Riley 2007 {published data only}
    1. Riley P, Vindigni S. Developing a nursing database system in Kenya. Health Services Research 2007;42(3p2):1389-405. [DOI: 10.1111/j.1475-6773.2007.00715.x] - DOI - PMC - PubMed
Ruton 2018 {published data only}
    1. Ruton H, Musabyimana A, Gaju E, Berhe A, Grépin KA, Ngenzi J, et al. The impact of an mHealth monitoring system on health care utilization by mothers and children: an evaluation using routine health information in Rwanda. Health Policy and Planning 2018;33(8):920-7. [DOI: ] - PMC - PubMed
Spero 2011 {published data only}
    1. Spero JC, McQuide PA, Matte R. Tracking and monitoring the health workforce: a new human resources information system (HRIS) in Uganda. Human Resources for Health 2011;9(6):1-10. [DOI: ] - PMC - PubMed
Stengel 2004 {published data only}
    1. Stengel D, Bauwens K, Walter M, Kopfer T, Ekkernkamp A. Comparison of handheld computer-assisted and conventional paper chart documentation of medical records. A randomized, controlled trial. Journal of Bone and Joint Surgery. American Volume 2004;86-a(3):553-60. - PubMed
Usman 2008 {published data only}
    1. Usman HR, Akhtar S, Habib F, Jehan I. Redesigned immunization card and center-based education to reduce childhood immunization dropouts in urban Pakistan: a randomized controlled trial. Vaccine 2009;27(3):467-72. [DOI: ] - PubMed
Valadez 2014 {published data only}
    1. Valadez JJ, Devkota B, Pradhan MM, Meherde P, Sonal GS, Dhariwal S, et al. Improving malaria treatment and prevention in India by aiding district managers to manage their programmes with local information: a trial assessing the impact of Lot Quality Assurance Sampling on programme outcomes. Tropical Medicine & International Health 2014;19(10):1226-36. [DOI: 10.1111/tmi.12354] - DOI - PubMed
Venkateswaren 2018 {published data only}
    1. Venkateswaran M, Mørkrid K, Ghanem B, Abbas E, Abuward I, Baniode M, et al. eRegQual—an electronic health registry with interactive checklists and clinical decision support for improving quality of antenatal care: study protocol for a cluster randomized trial. Trials 2018;19(54):1-12. [DOI: ] - PMC - PubMed
Waters 2013 {published data only}
    1. Waters KP, Zuber A, Willy RM, Kiriinya RN, Waudo AN, Oluoch T, et al. Kenya’s Health Workforce Information System: A model of impact on strategic human resources policy, planning and management. International Journal of Medical Informatics 2013;82(9):895-902. [DOI: ] - PubMed
Were 2010 {published data only}
    1. Were MC, Emenyonu N, Achieng M, Shen C, Ssali J, Mashaba JB, et al. Evaluating a scalable model for implementing electronic health records in resource-limited settings. Journal of the American Medical Informatics Association 2010;17(3):237-44. [DOI: https://doi.org/10.1136/jamia.2009.002303] - PMC - PubMed
Yen 2005 {unpublished data only}
    1. Yen PY, Gorman PN. Usability testing of digital pen and paper system in nursing documentation. AMIA Annual Symposium Proceeding Archive 2005;2005:844-8. - PMC - PubMed
Zurovac 2011 {published data only}
    1. Zurovac D, Sudoi RK, Akhwale WS, Ndiritu M, Hamer DH, Rowe AK, et al. The effect of mobile phone text-message reminders on Kenyan health workers' adherence to malaria treatment guidelines: a cluster randomised trial. Lancet 2011;378(9793):795-803. [DOI: ] - PMC - PubMed

References to studies awaiting assessment

He 2014 {published data only}
    1. He P, Yuan Z, Liu Y, Li G, Lv H, Yu H, et al. An evaluation of a tailored intervention on village doctors use of electronic health records. BMC Health Services Research 2014;14:217. [DOI: ] - PMC - PubMed
Monyarit 2014 {published data only}
    1. Monyarit S, Pan-ngum W, Lawpoolsri S, Yimsamran S, Pongnumkul S, Kaewkungwal J, et al. Advantages of using voiced questionnaire and image capture application for data collection from a minority group in rural areas along the Thailand-Myanmar border. Journal of Innovation in Health Informatics 2014;21(4):179-88. [DOI: ] - PubMed
O'Connor 2019 {published data only}
    1. O'Connor EC, Hutain J, Christensen M, Kamara MS, Conteh A, Sarriot E, et al. Piloting a participatory, community-based health information system for strengthening community-based health services: findings of a cluster-randomized controlled trial in the slums of Freetown, Sierra Leone. Journal of Global Health 2019;9(1):010418. [DOI: 10.7189/jogh.09.010418] - DOI - PMC - PubMed
Singh 2012 {published data only}
    1. Singh R, Anderson D, McLean-Plunkett E, Brooks R, Wisniewski A, Satchidanand N, et al. IT-enabled systems engineering approach to monitoring and reducing ADEs. American Journal of Managed Care 2012;18:169-75. - PubMed
Toda 2016 {published data only}
    1. Toda M, Njeru I, Zurovac D, Tipo SO, Kareko D, Mwau M, et al. Effectiveness of a mobile short-message-service-based disease outbreak alert system in Kenya. Emerging Infectious Diseases 2016;22(4):711-715. [DOI: 10.3201/eid2204.151459] - DOI - PMC - PubMed

Additional references

Agarwal 2019
    1. Agarwal S, Glenton C, Henschke N, Fonhus M, Lewin S Bergman H etal. Web Annex D: Tracking health commodity inventory and notifying stock levels via mobile devices (unpublished review). WHO guideline: Digital health recommendations for health system strengthening 2019.
Aqil 2009
    1. Aqil A, Lippeveld T, Hozumi D. PRISM framework: a paradigm shift for designing, strengthening and evaluating routine health information systems. Health Policy and Planning 2009;24(3):217-28. [DOI: ] - PMC - PubMed
Arah 2003
    1. Arah OA, Klazinga NS, Delnoij DM, ten Asbroek AH, Custers T. Conceptual frameworks for health systems performance: a quest for effectiveness, quality, and improvement. International Journal for Quality in Health Care 2003;15(5):377-98. [DOI: ] - PubMed
Aspry 2013
    1. Aspry KE, Furman R, Karalis DG, Jacobson TA, Zhang AM, Liptak GS, et al. Effect of health information technology interventions on lipid management in clinical practice: a systematic review of randomized controlled trials. Journal of Clinical Lipidology 2013;7(6):546-60. [DOI: ] - PubMed
Bassi 2010
    1. Bassi J, Lau F, Bardal S. Use of information technology in medication reconciliation: a scoping review. The Annals of Pharmacotherapy 2010;44(5):885-97. [DOI: ] - PubMed
Bassi 2012
    1. Bassi J, Lau F, Lesperance M. Perceived impact of electronic medical records in physician office practices: a review of survey-based research. Interactive Journal of Medical Research 2012;1(2):e3. [DOI: 10.2196/ijmr.2113] - DOI - PMC - PubMed
Bassi 2013
    1. Bassi J, Lau F. Measuring value for money: a scoping review on economic value of health information systems. Journal of the American Medical Informatics Association 2013;20(4):792-801. [DOI: ] - PMC - PubMed
Black 2011
    1. Black AD, Car J, Pagliari C, Anandan C, Cresswell K, Bokun T, et al. The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Medicine 2011;8(1):e1000387. [DOI: 10.1371/journal.pmed.1000387] - DOI - PMC - PubMed
Blaya 2010a
    1. Blaya J, Shin S, Yale G, Suarez C, Asencios L, Contreras C, et al. Electronic laboratory system reduces errors in National Tuberculosis Program: a cluster randomized controlled trial. International Journal of Tuberculosis and Lung Disease 2010;2214(8):1009–15. - PMC - PubMed
Blaya 2010b
    1. Blaya JA, Fraser HS, Holt B. E-health technologies show promise in developing countries. Health Affairs 2010;29(2):244-51. [DOI: ] - PubMed
Blaya 2011
    1. Blaya J, Shin S, Contreras C, Yale G, Suarez C, Asencios L, et al. Full impact of laboratory information system requires direct use by clinical staff: cluster randomized controlled trial. Journal of the American Medical Informatics Association 2011;18(1):11-6. [DOI: ] - PMC - PubMed
Boonstra 2010
    1. Boonstra A, Broekhuis M. Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions. BMC Health Services Research 2010;10:231. [DOI: ] - PMC - PubMed
Bosch‐Caplanch 2018
    1. Bosch-Caplanch X, Zuske M, Auer C. Effects of interventions to improve Health Information use Systems (draft). Swiss Tropical and Public Health Institute, Swiss Centre for International Health 2018.
Chaudhry 2006
    1. Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Annals of Internal Medicine 2006;144(10):742-52. [DOI: ] - PubMed
Covidence 2015 [Computer program]
    1. Covidence systematic review software https://www.covidence.org. Melbourne, Australia: Veritas Health Innovation, 2015.
DeLone 1992
    1. DeLone WH, McLean ER. Information system success: the quest for the dependent variable. Information Systems Research 1992;3(1):60-95. [DOI: ]
Dixon‐Woods 2013
    1. Dixon-Woods M, Redwood S, Leslie M, Minion J, Martin GP, Coleman JJ. Improving quality and safety of care using "techno-vigilance": an ethnographic case study of secondary data from an electronic prescribing and decision support system. Milbank Quarterly 2013;91(3):424-54. [10.1111/1468-0009.12021] - PMC - PubMed
EPOC 2017a
    1. Effective Practice and Organisation of Care (EPOC). What study designs should be included in an EPOC review and what should they be called? EPOC-specific resources for review authors. Oslo: Norwegian Knowledge Centre for the Health Services; 2017. https://epoc.cochrane.org/sites/epoc.cochrane.org/files/public/uploads/R... (accessed 16 December 2015).
EPOC 2017b
    1. Effective Practice and Organisation of Care (EPOC). What outcomes should be reported in EPOC reviews? EPOC resources for review authors. Oslo: Norwegian Knowledge Centre for the Health Services; 2017. https://epoc.cochrane.org/sites/epoc.cochrane.org/files/public/uploads/R... (accessed 16 December 2015).
EPOC 2017c
    1. Effective Practice and Organisation of Care (EPOC). Suggested risk of bias criteria for EPOC reviews. EPOC Resources for review authors. Oslo: Norwegian Knowledge Centre for the Health Services; 2017. https://epoc.cochrane.org/sites/epoc.cochrane.org/files/public/uploads/R... (accessed 16 December 2015).
EPOC 2017d
    1. Effective Practice and Organisation of Care (EPOC). Synthesising results when it does not make sense to do a meta-analysis. EPOC Resources for review authors. Oslo: Norwegian Knowledge Centre for the Health Services; 2017. https://epoc.cochrane.org/sites/epoc.cochrane.org/files/public/uploads/R... (accessed 16 December 2015).
GRADEpro GDT [Computer program]
    1. GRADEpro Guideline Development Tool. McMaster University (developed by Evidence Prime, Inc.). Available from www.gradepro.org, 2015.
Guyatt 2008
    1. Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schünemann HJ, GRADE Working Group. What is "quality of evidence" and why is it important to clinicians? British Medical Journal 2008;336(7651):995-8. [DOI: 10.1136/bmj.39490.551019.BE] - DOI - PMC - PubMed
Hazel 2017
    1. Hazel E, Chimbalanga E, Chimuna T, Nsona H, Mtimuni A, Kaludzu E, et al. Using data to improve programs: assessment of a data quality and use intervention package for Integrated Community Case Management in Malawi. Global Health: Science and Practice 2017;5(3):355-66. [DOI: ] - PMC - PubMed
Higgins 2019
    1. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.0 (updated July 2019). Available from www.training.cochrane.org/handbook.org 2019. - PMC - PubMed
Hotchkiss 2010
    1. Hotchkiss DR, Aqil A, Lippeveld T, Mukooyo E. Evaluation of the Performance of Routine Information System Management (PRISM) framework: evidence from Uganda. BMC Health Services Research 2010;10(188):1-17. [DOI: ] - PMC - PubMed
Hotchkiss 2012
    1. Hotchkiss D, Diana M, Foreit K. How can routine health information systems improve health systems in low-resource settings? Assessing the evidence base. Measure Evaluation Special Report. Carolina Population Center, University of North Carolina at Chapel Hill. http://www.hrhresourcecenter.org/node/4089 (accessed 16 December 2015):1-50.
Jordan 2010
    1. Jordan M, Jordon Lanham H, Anderson RA, McDaniel RR Jr. Implications of complex adaptive systems theory for interpreting research about health care organization. Journal of Evaluation in Clinical Practice 2010;16(1):228-31. [DOI: 10.1111/j.1365-2753.2009.01359.x] - DOI - PMC - PubMed
Kebede 2010
    1. Kebede D, Zielinski C, Ebongue Mbondji P, Sanou I, Asamoah-Odei E, Soumbey-Alley EW, et al. Improving the availability, quality and use of health information, research evidence and knowledge to strengthen health systems. African Health Monitor, World Health Organization Regional Office for Africa 2010;(12):53-64. [https://www.afro.who.int/sites/default/files/2017-06/Improving_Availabil...]
Lau 2010
    1. Lau F, Kuziemsky C, Price M, Gardner J. A review on systematic reviews of health information system studies. Journal of the American Medical Informatics Association 2010;17(6):637-45. [DOI: ] - PMC - PubMed
Leatherman 2010
    1. Leatherman S, Ferris T, Berwick D, Omaswa F, Crisp N. The role of quality improvement in strengthening health systems in developing countries. International Journal for Quality in Health Care 2010;22(4):237-43. [DOI: ] - PubMed
Lewin 2006
    1. Lewin SA, Dick J, Pond P, Zwarenstein M, Aja G, Wyk B, et al. Lay health workers in primary and community health care. Cochrane Database of Systematic Reviews 2006, Issue 1. Art. No: CD004015. [DOI: 10.1002/14651858.CD004015.pub2] - DOI - PubMed
Liberati 2009
    1. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Journal of Clinical Epidemiology 2009;62(10):e1-e34. [DOI: ] - PubMed
Lippeveld 1997
    1. Lippeveld T, Sauerborn R, Sapirie S. Health information systems: Making them work. World Health Forum, World Health Organization 1997;18(2):176-84. - PubMed
Lippeveld 2000
    1. Lippeveld T, Sauerborn R, Bodart C. Design and Implementation of Health Information Systems. Geneva: World Health Organization, 2000. [://scholar.google.com/scholar?hl=en&as_sdt=0%2C47&q=Design+and+Impl...
Littlejohns 2003
    1. Littlejohns P, Wyatt JC, Garvican L. Evaluating computerised health information systems: hard lessons still to be learnt. British Medical Journal 2003;326:860-3. [DOI: https://doi.org/10.1136/bmj.326.7394.860 ] - PMC - PubMed
Mutale 2013
    1. Mutale W, Chintu N, Amoroso C, Awoonor-Williams K, Phillips J, Baynes C, et al. Improving health information systems for decision making across five sub-Saharan African countries: implementation strategies from the African Health Initiative. BMC Health Services Research 2013;13(Suppl 2):S9. [DOI: ] - PMC - PubMed
Nutley 2013
    1. Nutley T, Reynolds HW. Improving the use of health data for health system strengthening. Global Health Action 2013;6(1):1-10. [DOI: ] - PMC - PubMed
Nutley 2014
    1. Nutley T, Gnassou L, Traore M, Bosso AE, Mullen S. Moving data off the shelf and into action: an intervention to improve data-informed decision making in Cote d'Ivoire. Global Health Action 2014;7:25035. [PMID: ] - PMC - PubMed
Rahimi 2007
    1. Rahimi B, Vimarlund V. Methods to evaluate health information systems in healthcare settings: a literature review. Journal of Medical Systems 2007;31(5):397-432. [DOI: ] - PubMed
Rahimi 2009
    1. Rahimi B, Vimarlund V, Timpka T. Health information system implementation: a qualitative meta-analysis. Journal of Medical Systems 2009;33(5):359-68. - PubMed
Ramsay 2003
    1. Ramsay CR, Matowe L, Grilli R, Grimshaw JM, Thomas RE. Interrupted time series designs in health technology assessment: lessons from two systematic reviews of behavior change strategies. International Journal of Technology Assessment in Health Care 2003;19(4):613-23. [DOI: https://doi.org/10.1017/S0266462303000576] - PubMed
Review Manager 2014 [Computer program]
    1. Review Manager 5 (RevMan 5). Version 5.3. Copenhagen: Nordic Cochrane Centre, The Cochrane Collaboration, 2014.
Riley 2012
    1. Riley PL, Zuber A, Vindigni SM, Gupta N, Verani AR, Sunderland NL, et al. Information systems in human resources for health: a global view. Human Resources for Health 2012;10:7. [DOI: ] - PMC - PubMed
Shieshia 2014
    1. Shieshia M, Noel M, Andersson S, Felling B, Alva S, Agarwal S, et al. Strengthening community health supply chain performance through an integrated approach: using mHealth technology and multilevel teams in Malawi. Journal of Global Health 2014;4(2):1-12. [DOI: 10.7189/jogh.04.020406] - DOI - PMC - PubMed
Sligo 2017
    1. Sligo J, Gauld R, Roberts V, Villa L. A literature review for large-scale health information system project planning, implementation and evaluation. International Journal of Medical Informatics 2017;97:86-97. [DOI: ] - PubMed
Stringer 2013
    1. Stringer JA, Chisembele-Taylor A, Chibwesha CJ, Chi HF, Ayles H, Manda, H, et al. Protocol-driven primary care and community linkages to improve population health in rural Zambia: the Better Health Outcomes through Mentoring and Assessment (BHOMA) project. BMC Health Services Research 2013;13(2):S7. [DOI: ] - PMC - PubMed
Tursunbayeva 2017
    1. Tursunbayeva A, Bunduchi R, Franco M, Pagliari C. Human resource information systems in health care: a systematic evidence review. Journal of the American Medical Informatics Association : JAMIA 2017;24(3):633-54. [DOI: ] - PMC - PubMed
Wagenaar 2017
    1. Wagenaar BH, Hirschhorn LR, Henley C, Gremu A, Sindano N, Chilengi R, the AHI PHIT Partnership Collaborative. Data-driven quality improvement in low-and middle-income country health systems: lessons from seven years of implementation experience across Mozambique, Rwanda, and Zambia. BMC Health Services Research 2017;17(3):830. [DOI: 10.1186/s12913-017-2661-x] - DOI - PMC - PubMed
WHO 2007
    1. World Health Organization. Everybody's business. Strengthening health systems to improve health outcomes: WHO’s framework for action. World Health Organization 2015. [https://apps.who.int/iris/bitstream/handle/10665/43918/9789241596077_eng...]
WHO 2008
    1. World Health Organization. Framework and standards for country health information systems (Second edition). Health Metrics Network, World Health Organization 2008:https://apps.who.int/iris/bitstream/handle/10665/43872/9789244595947_rus....
WHO 2010
    1. World Health Organization. Monitoring the building blocks of health systems: a handbook of indicators and their measurement strategies. World Health Organisation 2010:http://www.who.int/healthinfo/systems/monitoring/en/. [http://www.who.int/healthinfo/systems/monitoring/en/]
WHO 2017
    1. World Health Organization. National health workforce accounts: a handbook. World Health Organisation 2017. [https://apps.who.int/iris/bitstream/handle/10665/259360/9789241513111-en...]
WHO 2019
    1. World Health Organization. WHO Guideline: recommendations on digital interventions for health system strengthening. World Health Organisation 2019:https://www.who.int/reproductivehealth/publications/digital-intervention.... [https://www.who.int/reproductivehealth/publications/digital-intervention...] - PubMed
Willis 2013
    1. Willis CD, Riley BL, Herbert CP, Best A. Networks to strengthen health systems for chronic disease prevention. American Journal of Public Health 2013;103(11):e39-48. [DOI: ] - PMC - PubMed
Zuske 2017
    1. Zuske M, Jarrett C, Christian Auer C, Bosch-Capblanch X. Health information use systems: framework synthesis (draft). Swiss Tropical and Public Health Institute, Swiss Centre for International Health 2017.

References to other published versions of this review

Leon 2015
    1. Leon N, Brady L, Kwamie A, Daniels K. Routine Health Information System (RHIS) interventions to improve health systems management. Cochrane Database of Systematic Reviews 2015, Issue 12. Art. No: CD012012. [DOI: 10.1002/14651858.CD012012] - DOI - PMC - PubMed

Publication types

MeSH terms

Substances