Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2016 Dec 2;12(12):CD008992.
doi: 10.1002/14651858.CD008992.pub3.

External inspection of compliance with standards for improved healthcare outcomes

Gerd Flodgren  1 , Daniela C Gonçalves-Bradley  2 , Marie-Pascale Pomey  3
Affiliations

External inspection of compliance with standards for improved healthcare outcomes

Gerd Flodgren et al. Cochrane Database Syst Rev. .

Abstract

Background: Inspection systems are used in healthcare to promote quality improvements (i.e. to achieve changes in organisational structures or processes, healthcare provider behaviour and patient outcomes). These systems are based on the assumption that externally promoted adherence to evidence-based standards (through inspection/assessment) will result in higher quality of healthcare. However, the benefits of external inspection in terms of organisational-, provider- and patient-level outcomes are not clear. This is the first update of the original Cochrane review, published in 2011.

Objectives: To evaluate the effectiveness of external inspection of compliance with standards in improving healthcare organisation behaviour, healthcare professional behaviour and patient outcomes.

Search methods: We searched the following electronic databases for studies up to 1 June 2015: the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, Embase, Database of Abstracts of Reviews of Effectiveness, HMIC, ClinicalTrials.gov and the World Health Organization International Clinical Trials Registry Platform. There was no language restriction and we included studies regardless of publication status. We also searched the reference lists of included studies and contacted authors of relevant papers, accreditation bodies and the International Organization for Standardization (ISO), regarding any further published or unpublished work. We also searched an online database of systematic reviews (PDQ-evidence.org).

Selection criteria: We included randomised controlled trials (RCTs), non-randomised trials (NRCTs), interrupted time series (ITSs) and controlled before-after studies (CBAs) evaluating the effect of external inspection against external standards on healthcare organisation change, healthcare professional behaviour or patient outcomes in hospitals, primary healthcare organisations and other community-based healthcare organisations.

Data collection and analysis: Two review authors independently applied eligibility criteria, extracted data and assessed the risk of bias of each included study. Since meta-analysis was not possible, we produced a narrative results summary. We used the GRADE tool to assess the certainty of the evidence.

Main results: We did not identify any new eligible studies in this update. One cluster RCT involving 20 South African public hospitals and one ITS involving all acute hospital trusts in England, met the inclusion criteria. A trust is a National Health Service hospital which has opted to withdraw from local authority control and be managed by a trust instead.The cluster RCT reported mixed effects of external inspection on compliance with COHSASA (Council for Health Services Accreditation for South Africa) accreditation standards and eight indicators of hospital quality. Improved total compliance score with COHSASA accreditation standards was reported for 21/28 service elements: mean intervention effect was 30% (95% confidence interval (CI) 23% to 37%) (P < 0.001). The score increased from 48% to 78% in intervention hospitals, while remaining the same in control hospitals (43%). The median intervention effect for the indicators of hospital quality of care was 2.4% (range -1.9% to +11.8%).The ITS study evaluated compliance with policies to address healthcare-acquired infections and reported a mean reduction in MRSA (methicillin-resistant Staphylococcus aureus) infection rates of 100 cases per quarter (95% CI -221.0 to 21.5, P = 0.096) at three months' follow-up and an increase of 70 cases per quarter (95% CI -250.5 to 391.0; P = 0.632) at 24 months' follow-up. Regression analysis showed similar MRSA rates before and after the external inspection (difference in slope 24.27, 95% CI -10.4 to 58.9; P = 0.147).Neither included study reported data on unanticipated/adverse consequences or economic outcomes. The cluster RCT reported mainly outcomes related to healthcare organisation change, and no patient reported outcomes other than patient satisfaction.The certainty of the included evidence from both studies was very low. It is uncertain whether external inspection accreditation programmes lead to improved compliance with accreditation standards. It is also uncertain if external inspection infection programmes lead to improved compliance with standards, and if this in turn influences healthcare-acquired MRSA infection rates.

Authors' conclusions: The review highlights the paucity of high-quality controlled evaluations of the effectiveness and the cost-effectiveness of external inspection systems. If policy makers wish to understand the effectiveness of this type of intervention better, there needs to be further studies across a range of settings and contexts and studies reporting outcomes important to patients.

PubMed Disclaimer

Conflict of interest statement

GF state no conflict of interest.

DGB state no conflict of interest.

MPP state no conflict of interest.

Figures

1
1
Study flow diagram.
See this image and copyright information in PMC

Update of

References

References to studies included in this review

OPM 2009 {published data only}
    1. OPM evaluation team. Evaluation of the Healthcare Commission's Healthcare Associated Infections Inspection Programme. OPM Report 2009:1‐23.
Salmon 2003 {published data only}
    1. Salmon JW, Heavens J, Lombard C, Tavrow P. The impact of accreditation on the quality of hospital care: KwaZulu‐Natal Province, Republic of South Africa. Operations Research Results. Bethesda: U.S. Agency for International Development (USAID), Quality Assurance Project, University Research Co., LLC, 2003; Vol. 2, issue 17:1‐49.

References to studies excluded from this review

Al Tehewy 2009 {published data only}
    1. Al Tehewy N, Salem B, Habil I, Okda S. Evaluation of accreditation program in non‐governmental organisations' health units in Egypt: short term outcomes. International Journal for Quality in Healthcare 2009;21(3):183‐9. - PubMed
Brooke 2008 {published data only}
    1. Brooke BS, Perler BA, Dominici F, Makary MA, Pronovost PJ. Reduction of in‐hospital mortality among California hospitals meeting Leapfrog evidence‐based standards for abdominal aortic aneurysm repair. Journal of Vascular Surgery 2008;47(6):1155‐6; 1163‐4. - PubMed
Frasco 2005 {published data only}
    1. Frasco PE, Sprung J, Trentman TL. The impact of the joint commission for accreditation of healthcare organisational pain initiative on perioperative opiate consumption and recovery room length of stay. Anesthesia & Analgesia 2005;100:162‐8. - PubMed
Kowiatek 2002 {published data only}
    1. Kowiatek JG, Weber RJ, Schilling DE, McKaveney TP. Monitoring compliance with JCAHO standards using a medication‐control review tool. American Journal of Health System Pharmacy 2002;59(18):1763‐7. - PubMed
Laselle 2006 {published data only}
    1. Laselle TJ, May SK. Medication orders are written clearly and transcribed accurately? Implementing Medication Management Standard 3.20 and National Patient Safety Goal 2b. Hospital Pharmacy 2006;41:82‐7.
Mattes 1987 {published data only}
    1. Mattes JA. A controlled evaluation of a JCAH regulation. Psychiatric Hospital 1987;18(3):131‐3. - PubMed
OPM 2007 {published data only}
    1. Office of Public management (OPM). Evaluation of a national audit of specialist in healthcare services for people with learning difficulties in England. OPM Reports 2007.
OPM 2008a {published data only}
    1. Office of Public Management (OPM). Evaluation of the Healthcare Commission's Assessment Process 2006‐2007. OPM Reports 2008.
OPM 2008b {published data only}
    1. Office of Public Management (OPM). Evaluation of the Healthcare Commission investigation function. OPM Reports 2008.
Russell 2014 {published data only}
    1. Russell GK, Jimenez S, Martin L, Stanley R, Peake MD, Woolhouse I. A multicentre randomised controlled trial of reciprocal lung cancer peer review and supported quality improvement: results from the improving lung cancer outcomes project. British Journal of Cancer 2014;110(8):1936‐42. [DOI: 10.1038/bjc.2014.146] - DOI - PMC - PubMed
Shaw 2003 {published data only}
    1. Shaw CD. Measuring against clinical standards. Clinica Chimica Acta 2003;333(2):115‐24. - PubMed
Shonka 2009 {published data only}
    1. Shonka DC, Ghanem TA, Hubbard MA, Barker DA, Kesser BW. Four years of accreditation council of graduate medical education duty hour regulations: have they made a difference?. Laryngoscope 2009;119(4):635‐9. - PubMed
Walsh 1998 {published data only}
    1. Walsh N, Walshe K. Accreditation in primary care: an evaluation of the Royal College of General Practitioners' team based practice accreditation programme. University of Birmingham Health Services Management Centre, 1998.
Winchester 2008 {published data only}
    1. Winchester DP, Kaufman C, Anderson B, El‐Tamer M, Kurtzman SH, Masood S, et al. The National Accreditation Program for Breast Centers: quality improvement through interdisciplinary evaluation and management. Bulletin of the American College of Surgeons 2008;93(10):13‐7. - PubMed

References to studies awaiting assessment

Browne 2015 {published data only}
    1. Browne DL, Bamford M, Roe M, Harrington A, Paisey RB. A structured diabetic foot care peer review programme commissioned by the Strategic Clinical Network to assess clinical pathways across each clinical commissioning group (CCG) in southwest England and reduce diabetes amputation rates. Diabetic Medicine 2015;32(Suppl 1):197.

Additional references

Accreditation Canada 2015
    1. Nicklin W. The value and impact of healthcare accreditation: a literature review. Accreditation Canada 2015.
Care Quality Commission 2013
    1. Raising standards: putting people first. Our strategy for 2013‐2016. Care Quality Commission 2013.
Davis 2001
    1. Davis D, Downe J, Martin S. External inspection of local government: driving improvement or drowning in detail?. Joseph Rowntree Foundation 2001.
Department of Health 2006
    1. Department of Health. Code of practice for the prevention and control of healthcare associated infections. The Health Act 2006:1‐41.
EPOC 2009
    1. EPOC. Risk of bias tool. Available from epoc.cochrane.org/epoc‐resources‐review‐authors 2009.
EPOC 2015
    1. Effective Practice, Organisation of Care (EPOC). EPOC Resources for review authors. Available at: epoc.cochrane.org/epoc‐specific‐resources‐review‐authors. Oslo: Norwegian Knowledge Centre for the Health Services, 2015.
Greenfield 2008
    1. Greenfield D, Braithwaite J. Health sector accreditation research: a systematic review. International Journal for Quality in Health Care 2008;20(3):172‐83. - PubMed
Hart 2013
    1. Hart K, Djasri H, Utarini A. Regulating the quality of healthcare: lessons from hospital accreditation in Australia and Indonesia. Health Policy & Health Finance Hub. Working papers. 2013;28:1‐22.
Higgins 2011
    1. Higgins JPT, Altman DG (editors). Chapter 8: Assessing risk of bias in included studies. In: Higgins JP, Green S, editor(s). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from www.cochrane‐handbook.org.
Hopewell 2009
    1. Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K. Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database of Systematic Reviews 2009, Issue 1. [DOI: 10.1002/14651858.MR000006.pub3] - DOI - PMC - PubMed
Hovlid 2015
    1. Hovlid E, Høifødt H, Smedbråten B, Braut GS. A retrospective review of how nonconformities are expressed and finalized in external inspections of health‐care facilities. BMC Health Service Research 2015;15(405):1‐11. - PMC - PubMed
ISO 2004
    1. International Organization for Standardization/International Electrotechnical Commission. Standardization and related activities. ISO/IEC Guide 2004; Vol. 2.
Jamtvedt 2006
    1. Jamtvedt G, Young JM, Kristoffersen DT, O'Brien MA, Oxman AD. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews 2006, Issue 2. [DOI: 10.1002/14651858.CD000259.pub2] - DOI - PubMed
Moher 2009
    1. Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta‐Analyses: the PRISMA Statement. PLoS Medicine 2009;6(6):e1000097. - PMC - PubMed
Pomey 2005
    1. Pomey MP, François P, Contandriopoulos AP, Tosh A, Bertrand D. Paradoxes in French Accreditation. Quality and Safety in Health Care 2005;14(1):51‐5. - PMC - PubMed
Pomey 2010
    1. Pomey MP, Lemieux‐Charles L, Champagne F, Angus D, Shabah A, Contandriopoulos AP. Does accreditation stimulate change? A study of the impact of the accreditation process on Canadian healthcare organizations. Implementation Science 2010;26(5):31. - PMC - PubMed
RevMan 2014 [Computer program]
    1. The Nordic Cochrane Centre, The Cochrane Collaboration. Review Manager (RevMan). Version 5.3. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2014.
Shaw 2001
    1. Shaw C. External assessment of healthcare. British Medical Journal 2001;322:851‐4.
Shaw 2004
    1. Shaw C. The external assessment of health services. Worlds Hospitals and Health Services 2004;40(1):24‐7. - PubMed
Shaw 2013
    1. Shaw CD, Braithwaite J, Moldovan M, Nicklin W, Grgic I, Fortune T, et al. Profiling health‐care accreditation organizations: an international survey. International Journey of Quality in Health Care 2013;25(3):222‐31. - PubMed
Tuijn 2011
    1. Tuijn SM, Robben PB, Janssens FJ, Bergh H. Evaluating instruments for regulation of health care in the Netherlands. Journal of Evaluation of Clinical Practice 2011;17(3):411‐9. - PubMed
Walsche 2000
    1. Walsche K, Freeman T, Latham L, Wallace L, Spurgeon P. Chapter 6. The development of external reviews of clinical governance. Clinical Governance ‐ from Policy to Practice. Birmingham, UK: University of Birmingham, Health Services Management Centre, 2000.

References to other published versions of this review

Flodgren 2011
    1. Flodgren G, Pomey M‐P, Taber SA, Eccles MP. Effectiveness of external inspection of compliance with standards in improving healthcare organisation behaviour, healthcare professional behaviour or patient outcomes. Cochrane Database of Systematic Reviews 2011, Issue 11. [DOI: 10.1002/14651858.CD008992.pub2] - DOI - PMC - PubMed

Publication types

MeSH terms

LinkOut - more resources