Cleft Lip Repair Competence Can Be Evaluated with High-fidelity Simulation

Abstract

Background: Demonstrating competency before independent practice is increasingly important in surgery. This study tests the hypothesis that a high-fidelity cleft lip simulator can be used to discriminate performance between training levels, demonstrating its utility for assessing procedural competence.

Methods: During this prospective cohort study, participants performed a unilateral cleft lip repair on a high-fidelity simulator. Videos were blindly rated using the Objective Structured Assessment of Technical Skills (OSATS) and the Unilateral Cleft Lip Repair Competency Assessment Tool (UCLR). Digital measurement of symmetry was estimated. Influence of training level and cumulative prior experience on each score was estimated using Pearson r.

Results: Participants (n = 26) ranged from postgraduate year 3 to craniofacial fellow. Training level correlated best with UCLR (R = 0.4842, P = 0.0122*) and more weakly with OSATS (R = 0.3645, P = 0.0671), whereas cumulative prior experience only weakly correlated with UCLR (R = 0.3450, P = 0.0843) and not with OSATS (R = 0.1609, P = 0.4323). UCLR subscores indicated marking the repair had little correlation with training level (R = 0.2802, P = 0.1656), whereas performance and result did (R = 0.5152, P = 0.0071*, R = 0.4226, P = 0.0315*, respectively). Correlation between symmetry measures and training level was weak.

Conclusions: High-fidelity simulation paired with an appropriate procedure-specific assessment tool has the construct validity to evaluate performance for cleft lip repair. Simply being able to mark a cleft lip repair is not an accurate independent assessment method nor is symmetry of the final result.

Fig. 1.

Frontal view used for rating simulated cleft lip repair videos.

Fig. 2.

Example symmetry measurements. These color maps are examples of the output from 3-matic after overlaying a 3D scan of the repaired simulator with its mirror image. Areas of overlap between the two images are represented in green, while increasing distance between the two images caused by asymmetry shifts the color from green to yellow to red (scale in mm at far right). A more symmetrical or better cleft lip repair is shown on the left with a smaller median distance (0.8674 mm) and a smaller RMS value (1.1242 mm). A more asymmetrical or worse cleft lip repair is shown on the right with a larger median distance (1.1458 mm) and larger RMSD (1.4742 mm).

Fig. 3.

Correlation between structured rating scales and participant training level. A, Relationship between training level and OSATS. B, Relationship between training level and total UCLR score. C, Relationship between training level and UCLR Marking subscore. D, Relationship between training level and UCLR Performing subscore. E, Relationship between training level and UCLR result subscore.

Fig. 4.

Correlation between structured rating scales and participant total prior experience with cleft lip repair. A, Relationship between prior experience and OSATS. B, Relationship between prior experience and total UCLR score. C, Relationship between prior experience and UCLR marking subscore. D, Relationship between prior experience and UCLR performing subscore. E, Relationship between prior experience and UCLR result subscore.