Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Jun 1;155(6):486-492.
doi: 10.1001/jamasurg.2020.0424.

Development and Assessment of the Wisconsin Surgical Coaching Rubric

Kara A Vande Walle  1 , Sudha R Pavuluri Quamme  1 , Heather L Beasley  1 , Glen E Leverson  1 , Hala N Ghousseini  2 , Janet C Dombrowski  3 , Brian T Fry  4 , Justin B Dimick  4   5 , Douglas A Wiegmann  1   6 , Caprice C Greenberg  1
Affiliations

Development and Assessment of the Wisconsin Surgical Coaching Rubric

Kara A Vande Walle et al. JAMA Surg. .

Abstract

Importance: Surgical coaching continues to gain momentum as an innovative method for continuous professional development. A tool to measure the performance of a surgical coach is needed to provide formative feedback to coaches for continued skill development and to assess the fidelity of a coaching intervention for future research and dissemination.

Objective: To evaluate the validity of the Wisconsin Surgical Coaching Rubric (WiSCoR), a novel tool to assess the performance of a peer surgical coach.

Design, setting, and participants: Surgical coaching sessions from November 2014 through February 2018 conducted by 2 statewide peer surgical coaching programs were audio recorded and transcribed. Twelve raters used WiSCoR to rate the performance of the surgical coach for each session. The study included peer surgical coaches in the Wisconsin Surgical Coaching Program (n = 8) and the Michigan Bariatric Surgery Collaborative coaching program (n = 15). The data were analyzed in 2019.

Interventions or exposures: Use of WiSCoR to rate peer surgical coaching sessions.

Main outcomes and measures: There were 282 WiSCoR ratings from the 106 coaching sessions included in the study. WiSCoR was evaluated using a framework, including inter-rater reliability assessed with Gwet weighted agreement coefficent. Descriptive statistics of WiSCoR were calculated.

Results: Eight coaches (35%) and 11 coachees (29%) were from the Wisconsin Surgical Program and 15 coaches (65%) and 27 coachees (71%) were from the Michigan Bariatric Surgery Collaborative. The validity of WiSCoR is supported by high interrater reliability (Gwet weighted agreement coefficient, 0.87) as well as a weakly positive correlation of WiSCoR to coachee ratings of coaches (r = 0.22; P = .04), rigorous content development, consistent rater training, and the association of WiSCoR with coach and coaching program development. The mean (SD) overall coach performance rating using WiSCoR was 3.23 (0.82; range, 1-5).

Conclusions and relevance: WiSCoR is a reliable measure that can assess the performance of a surgical coach, inform fidelity to coaching principles, and provide formative feedback to surgical coaches. While coachee ratings may reflect coachee satisfaction, they are not able to determine the quality of a coach.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest Disclosures: Dr Dimick reported personal fees from ArborMetrix, Inc during the conduct of the study. Dr Wiegmann reported personal fees from HFACS, Inc outside the submitted work. Dr Greenberg reported grants from Wisconsin Partnership Program and the National Institutes of Health/National Institute of Diabetes and Digestive and Kidney Diseases (NIH/NIDDK) during the conduct of the study and serving on the Johnson and Johnson Institute's Global Education Council and as president/founder of the Institute for Surgical Coaching. She is not paid for either of these positions. No other disclosures were reported.

Figures

Figure 1.
Figure 1.. Wisconsin Surgical Coaching Rubric (WiSCoR)
Figure 2.
Figure 2.. Scatterplot of Average Overall Wisconsin Surgical Coaching Rubric (WiSCoR) Ratings and Coachee Ratings of Coaches
See this image and copyright information in PMC

Comment in

References

    1. Greenberg CC, Ghousseini HN, Pavuluri Quamme SR, et al. ; Wisconsin Surgical Coaching Program . A statewide surgical coaching program provides opportunity for continuous professional development. Ann Surg. 2018;267(5):868-873. doi: 10.1097/SLA.0000000000002341 - DOI - PubMed
    1. Greenberg CC, Ghousseini HN, Pavuluri Quamme SR, Beasley HL, Wiegmann DA. Surgical coaching for individual performance improvement. Ann Surg. 2015;261(1):32-34. doi: 10.1097/SLA.0000000000000776 - DOI - PubMed
    1. Hu YY, Peyre SE, Arriaga AF, et al. Postgame analysis: using video-based coaching for continuous professional development. J Am Coll Surg. 2012;214(1):115-124. doi: 10.1016/j.jamcollsurg.2011.10.009 - DOI - PMC - PubMed
    1. Shubeck SP, Kanters AE, Sandhu G, Greenberg CC, Dimick JB. Dynamics within peer-to-peer surgical coaching relationships: early evidence from the Michigan Bariatric Surgical Collaborative. Surgery. 2018;164(2):185-188. doi: 10.1016/j.surg.2018.03.009 - DOI - PMC - PubMed
    1. Cook DA, Beckman TJ. Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med. 2006;119(2):166.e7-166.e16. doi: 10.1016/j.amjmed.2005.10.036 - DOI - PubMed

Publication types