See One, Do One, Forget One: Early Skill Decay After Paracentesis Training

Abstract

Introduction: Internal medicine residents perform paracentesis, but programs lack standard methods for assessing competence or maintenance of competence and instead rely on number of procedures completed. This study describes differences in resident competence in paracentesis over time.

Methods: From 2016 to 2017, internal medicine residents (n = 118) underwent paracentesis simulation training. Competence was assessed using the Paracentesis Competency Assessment Tool (PCAT), which combines a checklist, global scale, and entrustment score. The PCAT also delineates two categorical cut-point scores: the Minimum Passing Standard (MPS) and the Unsupervised Practice Standard (UPS). Residents were randomized to return to the simulation lab at 3 and 6 months (group A, n = 60) or only 6 months (group B, n = 58). At each session, faculty raters assessed resident performance. Data were analyzed to compare resident performance at each session compared with initial training scores, and performance between groups at 6 months.

Results: After initial training, all residents met the MPS. The number achieving UPS did not differ between groups: group A = 24 (40%), group B = 20 (34.5%), p = 0.67. When group A was retested at 3 months, performance on each PCAT component significantly declined, as did the proportion of residents meeting the MPS and UPS. At the 6-month test, residents in group A performed significantly better than residents in group B, with 52 (89.7%) and 20 (34.5%) achieving the MPS and UPS, respectively, in group A compared with 25 (46.3%) and 2 (3.70%) in group B (p < .001 for both comparison).

Discussion: Skill in paracentesis declines as early as 3 months after training. However, retraining may help interrupt skill decay. Only a small proportion of residents met the UPS 6 months after training. This suggests using the PCAT to objectively measure competence would reclassify residents from being permitted to perform paracentesis independently to needing further supervision.

Keywords: assessment; competency-based medical education; paracentesis; procedures; skill decay.

Figure 1

Flow chart of study process.

Figure 2

Graphical representation of decline in those achieving either the Minimum Passing Standard (a) or the Unsupervised Practice Standard (b) between groups over time. a Comparison between those achieving the Minimum Passing Standard (MPS) between training groups after initial training and at the 3-and 6-month tests. All residents achieved the MPS after initial training. At the 3-month test (group A only), a significantly lower proportion of residents achieved the MPS compared with initial training (n = 31, 52.5%, p < .001). At the 6-month test, a higher proportion of learners in group A retained criteria for MPS (n = 52, 89.7%) than those in group B (n = 25, 46.3%; p < .001). b Comparison between those achieving the Unsupervised Practice Standard (UPS) between training groups after initial training and at the 3- and 6-month tests. The proportion of residents achieving UPS after initial training was not different between groups: group A, n = 24 (40%) compared with group B, n = 20 (34.5%), p = 0.67. At the 3-month test (group A only), a significantly lower proportion of residents achieved the UPS compared with initial training (n = 8, 13.6%, p = .002). At the 6-month test, a higher proportion of learners in group A retained criteria for UPS (n = 20, 34.5%) than those in group B (n = 2, 3.7%; p < .001). In general, the proportion of residents achieving UPS was low at all time points.