The performance of junior doctors in applying clinical pharmacology knowledge and prescribing skills to standardized clinical cases

Abstract

Aims: Recent studies suggest a worryingly high proportion of final year medical students and new doctors feel unprepared for effective and safe prescribing. Little research has been undertaken on UK junior doctors to see if these perceptions translate into unsafe prescribing practice. We aimed to measure the performance of foundation year 1 (FY1) doctors in applying clinical pharmacology and therapeutics (CPT) knowledge and prescribing skills using standardized clinical cases.

Methods: A subject matter expert (SME) panel constructed a blueprint, and from these, twelve assessments focusing on areas posing high risk to patient safety and deemed as essential for FY1 doctors to know were chosen. Assessments comprised six extended matching questions (EMQs) and six written unobserved structured clinical examinations (WUSCEs) covering seven CPT domains. Two of each assessment types were administered over three time points to 128 FY1 doctors.

Results: The twelve assessments were valid and statistically reliable. Across seven CPT areas tested 51-75% of FY1 doctors failed EMQs and 27-70% failed WUSCEs. The WUSCEs showed three performance trends; 30% of FY1 doctors consistently performing poorly, 50% performing around the passing score, and 20% performing consistently well. Categorical rating of the WUSCEs revealed 5% (8/161) of scripts contained errors deemed as potentially lethal.

Conclusions: This study showed that a large proportion of FY1 doctors failed to demonstrate the level of CPT knowledge and prescribing ability required at this stage of their careers. We identified areas of performance weakness that posed high risk to patient safety and suggested ways to improve the prescribing by FY1 doctors.

Figure 1

The spread of percentage scores obtained by individual FY1 doctors on each EMQ and the relationship to the mean average percentage score and pass mark (70%). Confidence intervals are not shown in this figure. Key: = Pass mark of assessment item = Mean average percentage mark obtained by FY1 on each assessment item EMQ 1 topic = Contra-indications in the use of anti-coagulates and antibiotics EMQ 2 topic = Anti-coagulants and analgesics in the post operative patient EMQ 3 topic = Diverticular disease; i.v. administration of antibiotics EMQ 4 topic = Respiratory; Chronic obstructive pulmonary disease EMQ 5 topic = Diabetes; insulin selection EMQ 6 topic = Analgesia in emergency medicine Numbers in brackets e.g. (1), indicate the number of individual FY1 doctors who obtained the particular score on each question Number of FY1 doctors completing the assessment item; EMQ 1 =35, EMQ 2 = 42, EMQ 3 = 49, EMQ 4 = 30, EMQ 5 = 28, EMQ 6 = 26. Maximum possible scores for each question are: EMQ 1 = 8, EMQ 2 = 8, EMQ 3 = 8, EMQ 4 = 8, EMQ 5 = 10, EMQ 6 = 8. Minimum possible scores for each question are: EMQ 1 =−8, EMQ 2 =−8, EMQ 3 =−8, EMQ 4 =−8, EMQ 5 =−10, EMQ 6 =−8. Range of scores on each question (min;max): EMQ 1 =−2 to 8, EMQ 2 = 2 to 7, EMQ 3 =−2 to 8, EMQ 4 = 0 to 8, EMQ 5 = 2 to 9, EMQ 6 = 2 to 7

Figure 2

The spread of percentage scores obtained by individual FY1 doctors across each WUSCE. The possible scores in each WUSCE vary and are given in the key (below). The figure also shows the relationship of the mean percentage average scores obtained and the pass percentage for each WUSCE. Confidence intervals are not shown in this figure. Key: = Pass mark of assessment item = Mean average mark obtained by FY1 on each assessment item WUSCE 5 topic = Contra-indications in the use of anti-coagulates and antibiotics WUSCE 3 topic = Anti-coagulants and analgesics in the post-operative patient WUSCE 4 topic = Diverticular disease; i.v. administration of antibiotics WUSCE 6 topic = Respiratory; Chronic obstructive pulmonary disease WUSCE 2 topic = Analgesia in emergency medicine WUSCE 1 topic = Renal failure; fluid management and artiopathy Number of FY1 doctors completing the assessment item = WUSCE 1 = 44, WUSCE 2 = 33, WUSCE 3 = 25, WUSCE 4 = 21, WUSCE 5 = 20, WUSCE 6 = 18. Maximum possible scores for each question are: WUSCE 1 = 69.8, WUSCE 2 = 43, WUSCE 3 = 44.5, WUSCE 4 = 14.9, WUSCE 5 = 46.5, WUSCE 6 = 21.6. Minimum possible scores for each question are: WUSCE 1 =−13, WUSCE 2 = 0, WUSCE 3 = 0, WUSCE 4 = 0, WUSCE 5 =−2, WUSCE 6 =−4. Range of scores on each question (max;min): WUSCE 1 = 6.0 to 45.5, WUSCE 2 = 3.3 to 25.5, WUSCE 3 = 4.0 to 33.1, WUSCE 4 = 0.8 to 14.9, WUSCE 5 = 16.0 to 42.5, WUSCE 6 = 5.2 to 19.8

Figure 3

The profiles which emerged from visual inspection of individual FY1 doctors percentage pass marks. ‘Best performers’ group constituted less than 20% of FY1 doctors. ‘Middle performers’ represented approximately 50% of the FY1 doctors. ‘Worst performers’ are made up of slightly more than 30% of FY1 doctors completing each WUSCE. Best performers (—); Middle performers (); Worst performers ()