Audit and feedback: effects on professional practice

Abstract

Background: Audit and feedback (A&F) is a widely used strategy to improve professional practice. This is supported by prior Cochrane reviews and behavioural theories describing how healthcare professionals are prompted to modify their practice when given data showing that their clinical practice is inconsistent with a desirable target. Yet there remains uncertainty regarding the effects of A&F on improving healthcare practice and the characteristics of A&F that lead to a greater impact.

Objectives: To assess the effects of A&F on the practice of healthcare professionals and to examine factors that may explain variation in the effectiveness of A&F.

Search methods: With the Cochrane Effective Practice and Organisation of Care (EPOC) group information scientist, we updated our search strategy to include studies published from 2010 to June 2020. Search updates were performed on 28 February 2019 and 11 June 2020. We searched MEDLINE (Ovid), Embase (Ovid), CINAHL (EBSCO), the Cochrane Library, clinicaltrials.gov (all dates to June 2020), WHO ICTRP (all dates to February Week 3 2019, no information available in 2020 due to COVID-19 pandemic). An updated search and duplicate screen was completed on February 14, 2022; studies that met inclusion criteria are included in the 'Studies awaiting classification' section.

Selection criteria: Randomised trials, including cluster-trials and cross-over and factorial designs, featuring A&F (defined as measurement of clinical performance over a specified period of time (audit) and provision of the resulting data to clinicians or clinical teams (feedback)) in any trial arm that reported objectively measured health professional practice outcomes.

Data collection and analysis: For this updated review, we re-extracted data for each study arm, including theory-informed variables regarding how the A&F was conducted and behaviour change techniques for each intervention, as well as study-level characteristics including risk of bias. For each study, we extracted outcome data for every healthcare professional practice targeted by A&F. All data were extracted by a minimum of two independent review authors. For studies with dichotomous outcomes that included arms with and without A&F, we calculated risk differences (RDs) (absolute difference between arms in proportion of desired practice completed) and also odds ratios (ORs). We synthesised the median RDs and interquartile ranges (IQRs) across all trials. We then conducted meta-analyses, accounting for multiple outcomes from a given study and weighted by effective sample size, using reported (or imputed, when necessary) intra-cluster correlation coefficients. Next, we explored the role of baseline performance, co-interventions, targeted behaviour, and study design factors on the estimated effects of A&F. Finally, we conducted exploratory meta-regressions to test preselected variables that might be associated with A&F effect size: characteristics of the audit (number of indicators, aggregation of data); delivery of the feedback (multi-modal format, local champion, nature of comparator, repeated delivery); and components supporting action (facilitation, provision of specific plans for improvement, co-development of action plans).

Main results: We included 292 studies with 678 arms; 133 (46%) had a low risk of bias, 41 (14%) unclear, and 113 (39%) had a high risk of bias. There were 26 (9%) studies conducted in low- or middle-income countries. In most studies (237, 81%), the recipients of A&F were physicians. Professional practices most commonly targeted in the studies were prescribing (138 studies, 47%) and test-ordering (103 studies, 35%). Most studies featured multifaceted interventions: the most common co-interventions were clinician education (377 study arms, 56%) and reminders (100 study arms, 15%). Forty-eight unique behaviour change techniques were identified within the study arms (mean 5.2, standard deviation 2.8, range 1 to 29). Synthesis of 558 dichotomous outcomes measuring professional practices from 177 studies testing A&F versus control revealed a median absolute improvement in desired practice of 2.7%, with an IQR of 0.0 to 8.6. Meta-analyses of these studies, accounting for multiple outcomes from the same study and weighting by effective sample size accounting for clustering, found a mean absolute increase in desired practice of 6.2% (95% confidence interval (CI) 4.1 to 8.2; moderate-certainty evidence) and an OR of 1.47 (95% CI 1.31 to 1.64; moderate-certainty evidence). Effects were similar for pre-planned subgroup analyses focused on prescribing and test-ordering outcomes. Lower baseline performance and increased number of co-interventions were both associated with larger intervention effects. Meta-regressions comparing the presence versus absence of specific A&F components to explore heterogeneity, accounting for baseline performance and number of co-interventions, suggested that A&F effects were greater with individual-recipient-level data rather than team-level data, comparing performance to top-peers or a benchmark, involving a local champion with whom the recipient had a relationship, using interactive modalities rather than just didactic or just written format, and with facilitation to support engagement, and action plans to improve performance. The meta-regressions did not find significant effects with the number of indicators in the audit, comparison to average performance of all peers, or co-development of action plans. Contrary to expectations, repeated delivery was associated with lower effect size. Direct comparisons from head-to-head trials support the use of peer-comparisons versus no comparison at all and the use of design elements in feedback that facilitate the identification and action of high-priority clinical items.

Authors' conclusions: A&F can be effective in improving professional practice, but effects vary in size. A&F is most often delivered along with co-interventions which can contribute additive effects. A&F may be most effective when designed to help recipients prioritise and take action on high-priority clinical issues and with the following characteristics: 1. targets important performance metrics where health professionals have substantial room for improvement (audit); 2. measures the individual recipient's practice, rather than their team or organisation (audit); 3. involves a local champion with an existing relationship with the recipient (feedback); 4. includes multiple, interactive modalities such as verbal and written (feedback); 5. compares performance to top peers or a benchmark (feedback); 6. facilitates engagement with the feedback (action); 7. features an actionable plan with specific advice for improvement (action). These conclusions require further confirmatory research; future research should focus on discerning ways to optimise the effectiveness of A&F interventions.