Self-assessment of team performance using T-NOTECHS in simulated pediatric trauma resuscitation is not consistent with expert assessment

Minna M Wieck¹, Cory McLaughlin², Todd P Chang³, Alyssa Rake⁴, Caron Park⁵, Christianne Lane⁶, Rita V Burke⁷, L Caulette Young⁸, Elizabeth A Cleek⁹, Inge Morton¹⁰, Catherine J Goodhue¹¹, Randall S Burd¹², Henri R Ford¹³, Jeffrey S Upperman¹⁴, Aaron R Jensen¹⁵

Affiliations

¹ Division of Pediatric Surgery, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: wieck@ohsu.edu.
² Division of Pediatric Surgery, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: cmclaughlin@chla.usc.edu.
³ Division of Emergency and Transport Medicine, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: tochang@chla.usc.edu.
⁴ Division of Anesthesiology and Critical Care Medicine, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA; Las Madrinas Simulation Research Laboratory, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: arake@chla.usc.edu.
⁵ Southern California Clinical Translational Science Institute (SC-CTSI), Department of Preventative Medicine, Keck School of Medicine, University of Southern California, Los Angeles, CA 90089, USA. Electronic address: caronpar@usc.edu.
⁶ Southern California Clinical Translational Science Institute (SC-CTSI), Department of Preventative Medicine, Keck School of Medicine, University of Southern California, Los Angeles, CA 90089, USA. Electronic address: christianne.lane@med.usc.edu.
⁷ Trauma Program, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA; Division of Pediatric Surgery, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: riburke@chla.usc.edu.
⁸ Division of Anesthesiology and Critical Care Medicine, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA; Las Madrinas Simulation Research Laboratory, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: lyoung@chla.usc.edu.
⁹ Trauma Program, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA; Division of Pediatric Surgery, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: elizabeth.cleek@marquette.edu.
¹⁰ Division of Emergency and Transport Medicine, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: imorton@chla.usc.edu.
¹¹ Trauma Program, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA; Division of Pediatric Surgery, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: cgoodhue@chla.usc.edu.
¹² Division of Trauma and Burn Surgery, Children's National Medical Center, Washington, DC 20010, USA. Electronic address: rburd@childrensnational.org.
¹³ Trauma Program, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA; Division of Pediatric Surgery, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: hford@chla.usc.edu.
¹⁴ Trauma Program, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA; Division of Pediatric Surgery, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: jupperman@chla.usc.edu.
¹⁵ Trauma Program, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA; Division of Pediatric Surgery, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: ajensen@chla.usc.edu.

PMID: 29366483
PMCID: PMC7169991
DOI: 10.1016/j.amjsurg.2018.01.010

Self-assessment of team performance using T-NOTECHS in simulated pediatric trauma resuscitation is not consistent with expert assessment

Minna M Wieck et al. Am J Surg. 2018 Sep.

. 2018 Sep;216(3):630-635.

doi: 10.1016/j.amjsurg.2018.01.010. Epub 2018 Feb 1.

Authors

Affiliations

¹ Division of Pediatric Surgery, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: wieck@ohsu.edu.
² Division of Pediatric Surgery, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: cmclaughlin@chla.usc.edu.
³ Division of Emergency and Transport Medicine, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: tochang@chla.usc.edu.
⁴ Division of Anesthesiology and Critical Care Medicine, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA; Las Madrinas Simulation Research Laboratory, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: arake@chla.usc.edu.
⁵ Southern California Clinical Translational Science Institute (SC-CTSI), Department of Preventative Medicine, Keck School of Medicine, University of Southern California, Los Angeles, CA 90089, USA. Electronic address: caronpar@usc.edu.
⁶ Southern California Clinical Translational Science Institute (SC-CTSI), Department of Preventative Medicine, Keck School of Medicine, University of Southern California, Los Angeles, CA 90089, USA. Electronic address: christianne.lane@med.usc.edu.
⁷ Trauma Program, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA; Division of Pediatric Surgery, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: riburke@chla.usc.edu.
⁸ Division of Anesthesiology and Critical Care Medicine, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA; Las Madrinas Simulation Research Laboratory, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: lyoung@chla.usc.edu.
⁹ Trauma Program, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA; Division of Pediatric Surgery, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: elizabeth.cleek@marquette.edu.
¹⁰ Division of Emergency and Transport Medicine, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: imorton@chla.usc.edu.
¹¹ Trauma Program, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA; Division of Pediatric Surgery, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: cgoodhue@chla.usc.edu.
¹² Division of Trauma and Burn Surgery, Children's National Medical Center, Washington, DC 20010, USA. Electronic address: rburd@childrensnational.org.
¹³ Trauma Program, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA; Division of Pediatric Surgery, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: hford@chla.usc.edu.
¹⁴ Trauma Program, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA; Division of Pediatric Surgery, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: jupperman@chla.usc.edu.
¹⁵ Trauma Program, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA; Division of Pediatric Surgery, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: ajensen@chla.usc.edu.

PMID: 29366483
PMCID: PMC7169991
DOI: 10.1016/j.amjsurg.2018.01.010

Abstract

Background: The Trauma NOn-TECHnical Skills (T-NOTECHS) tool has been used to assess teamwork in trauma resuscitation, but its reliability and validity for self-assessment is unknown. Our purpose was to determine the reliability and validity of self-administered T-NOTECHS in pediatric trauma resuscitation.

Methods: Simulated in situ resuscitations were evaluated using T-NOTECHS in real time by experts and immediately afterwards by team members. Reliability was analyzed with linear-weighted kappa and intra-class correlation. T-NOTECHS scores were compared between expert (gold-standard) and self-assessment.

Results: Fifteen simulations were examined. T-NOTECHS scores were similar between self- and expert assessment for leadership. Self-assessment scores were higher than expert for the other domains and total composite score. Inter-rater reliability for total score was similar between the two groups, but differences were observed in the domains.

Conclusions: Self-assessment is not interchangeable with expert rating when using T-NOTECHS. Future studies need to determine how self-assessment can be best utilized.

Level of evidence: Studies of diagnostic accuracy - Level 2.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest

None of the authors have any conflicts of interest to disclose.

Figures

**Figure 1:**
Scatterplot of mean scores comparing expert and team self-assessment for each of the five T-NOTECHS domains and total composite score (N=15 teams assessed). Error bars represent standard error for expert raters (vertical) and team member self-assessment (horizontal).

See this image and copyright information in PMC

References

1. Kohn L. To err is human: an interview with the Institute of Medicine’s Linda Kohn. Jt Comm J Qual Improv 2000;26(4):227–34. - PubMed
1. Gawande AA, Zinner MJ, Studdert DM, Brennan TA. Analysis of errors reported by surgeons at three teaching hospitals. Surgery 2003;133(6):614–21. - PubMed
1. Scott DJ, Dunnington GL. The new ACS/APDS Skills Curriculum: moving the learning curve out of the operating room. J Gastrointest Surg 2008;12(2):213–21. - PubMed
1. Knudson MM, Khaw L, Bullard MK, et al. Trauma training in simulation: translating skills from SIM time to real time. J Trauma 2008;64(2):255–63; discussion 63–4. - PubMed
1. Holcomb JB, Dumire RD, Crommett JW, et al. Evaluation of trauma team performance using an advanced human patient simulator for resuscitation training. J Trauma 2002;52(6):1078–85; discussion 85–6. - PubMed

Publication types

Actions
Actions

MeSH terms

Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions

Grants and funding

LinkOut - more resources

Full Text Sources
- Elsevier Science
Other Literature Sources
- scite Smart Citations
Research Materials
- NCI CPTC Antibody Characterization Program

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Self-assessment of team performance using T-NOTECHS in simulated pediatric trauma resuscitation is not consistent with expert assessment

Affiliations

Self-assessment of team performance using T-NOTECHS in simulated pediatric trauma resuscitation is not consistent with expert assessment

Authors

Affiliations

Abstract

Conflict of interest statement

Figures

References

Publication types

MeSH terms

Grants and funding

LinkOut - more resources

Full Text Sources

Other Literature Sources

Research Materials