Evaluating the effectiveness of 2 educational interventions in family practice

Abstract

Background: Structured feedback of information can produce change in physician behaviour. The objective of this study was to assess the effectiveness of 2 educational interventions for improving the quality of care provided by family physicians in Ontario: the Practice Assessment Report (PAR) and the Continuing Medical Education Plan (CMEP) with a follow-up visit by a mentor.

Methods: The study was a randomized controlled trial. Physicians in the control group received only the PAR, whereas those in the experimental group received the PAR, CMEP and mentor interventions. The participants were 56 family physicians and general practitioners (27 in the PAR group and 29 in the CMEP group) in southern Ontario who agreed to participate in the interventions and provide data. A total of 2395 patients randomly sampled from the practices returned questionnaires and consented to have their medical records abstracted. The outcome measures were global scores in 4 areas--quality of care, charting, prevention and overall use of medications--and patient ratings of satisfaction with care and preventive practices. The measures were applied at the beginning (phase 1) and end (phase 2) of the study.

Results: The mean global scores at the end of the study for the PAR group were 70.1% for quality of care, 84.7% for prevention, 77.7% for charting and 82.2% for overall use of medications. The corresponding scores for the CMEP group were 68.3%, 82.1%, 76.4% and 83.2%. In the patient satisfaction component, the personal care scores at phase 2 were 93.6% for the PAR group and 94.6% for the CMEP group. Examples of the scores for prevention for the PAR group were 98.3% for children's current immunization, 96.6% for blood pressure measured within the previous 5 years, 79.4% for referral of women of the appropriate age for mammography within the previous 2 years, and 58.4% for discussion about alcohol use. The corresponding scores for the CMEP group were 95.8%, 97.6%, 77.6% and 64.6%. The changes in mean scores between phase 1 and phase 2 ranged from -1.9 to 2.3 points. There were no significant differences between the 2 groups in phase 1 or phase 2 scores or in change in scores. A total of 64.3% of the physicians rated the PAR as useful, 26.5% found the CMEP to be useful, and 41.0% considered the mentor strategy to be a useful form of continuing medical education. Although changes in practice related to the PAR, CMEP or mentor were reported by some physicians, they were not related to chart audit or patient scores.

Interpretation: Educational interventions based on quality-of-care assessments and directed to global improvements in quality of care did not result in improvements in the outcome measures. Educational interventions may have to be targeted to specific areas of the practice, with physicians being monitored and receiving ongoing feedback on their performance.