Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Aug 16;8(5):1357-1364.
doi: 10.1002/lio2.1123. eCollection 2023 Oct.

Gathering validity evidence for a 3D-printed simulator for training of myringotomy and ventilation tube insertion

Michael Lüscher  1 , Lars Konge  2 , Peter Tingsgaard  3 , Thomas Qvist Barrett  4 , Steven Arild Wuyts Andersen  5
Affiliations

Gathering validity evidence for a 3D-printed simulator for training of myringotomy and ventilation tube insertion

Michael Lüscher et al. Laryngoscope Investig Otolaryngol. .

Abstract

Objectives: This study aimed to gather validity evidence according to Messick's framework for a novel 3D-printed simulator for myringotomy with ventilation tube insertion for use in technical skills training of otorhinolaryngology (ORL) residents.

Methods: The study included 15 junior ORL residents (trainees) and 13 experienced teaching otolaryngologists (experts). Experts and trainees first received an identically structured introduction to the procedure, simulator, and simulation setup. Five procedures performed by each participant were video-recorded and ordered randomly for blinded rating by two independent raters. The rating tools used were a global rating scale (GBRS) and a task-specific checklist. Validity evidence was collected according to Messick's framework. Differences in time consumption and performance scores were analyzed. Finally, a pass/fail standard was established using the contrasting groups' method.

Results: Trainees used significantly more time per procedure (109 s, 95% CI: 99-120) than experts (82 s, 95% CI: 71-93; p < .001). Adjusted for repetition and rater leniency, experts achieved an average GBRS score of 18.8 (95% CI: 18.3-19.2) out of 20 points, whereas trainees achieved an average of 17.1 points (95% CI: 16.6-17.5; p < .001). In contrast to the task-specific checklist, the GBRS score discriminated between repetition number and participant experience. The pass/fail standard for the GBRS was established at 18.4 points.

Conclusion: We established educational validity evidence for a novel 3D-printed model for simulation-based training of ventilation tube insertion and established a reliable pass/fail standard.

Level of evidence: 1b.

Keywords: education; myringotomy; simulation; training; validity evidence; ventilation tube insertion.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflicts of interest.

Figures

FIGURE 1
FIGURE 1
Left—Simulator for myringotomy with ventilation tube insertion, consisting of a 3D‐printed model with a silicone ear mold. Right—Cartridge for fast change of tympanic membrane.
FIGURE 2
FIGURE 2
Study flow chart.
FIGURE 3
FIGURE 3
Mean GBRS and passing performance. Top: means plot of global rating scale scores (estimated marginal means after adjustment for experience, procedure number, and rater leniency) with 95% confidence intervals based on linear mixed models of total score. Bottom: Bar charts of % of passing performances for each group using the standard pass/fail level set at 18.4 point using the contrasting groups' method. GBRS, global rating scale.
FIGURE 4
FIGURE 4
Contrasting groups for standard setting.
See this image and copyright information in PMC

References

    1. Carraccio C, Englander R, Van Melle E, et al. Advancing competency‐based medical education: a charter for clinician‐educators. Acad Med J Assoc Am Med Coll. 2016;91(5):645‐649. - PubMed
    1. Shah N, Desai C, Jorwekar G, Badyal D, Singh T. Competency‐based medical education: an overview and application in pharmacology. Indian J Pharmacol. 2016;48(Suppl 1):S5‐S9. - PMC - PubMed
    1. Ten Cate O. Competency‐based postgraduate medical education: past, present and future. GMS J Med Educ. 2017;34(5):Doc69. - PMC - PubMed
    1. Bjerrum F, Sorensen JL, Thinggaard E, Strandbygaard J, Konge L. Implementation of a cross‐specialty training program in basic laparoscopy. JSLS. 2015;19(4):e2015.00059. - PMC - PubMed
    1. Bewley WL, O'Neil HF. Evaluation of medical simulations. Mil Med. 2013;178(10 Suppl):64‐75. - PubMed

LinkOut - more resources