Comparative Study

. 2020 Nov;95(11S Association of American Medical Colleges Learn Serve Lead: Proceedings of the 59th Annual Research in Medical Education Presentations):S95-S102.

doi: 10.1097/ACM.0000000000003638.

Shared Mental Models Among Clinical Competency Committees in the Context of Time-Variable, Competency-Based Advancement to Residency

Alan Schwartz¹, Dorene F Balmer², Emily Borman-Shoap³, Alan Chin⁴, Duncan Henry⁵, Bruce E Herman⁶, Patricia Hobday⁷, James H Lee⁸, Sara Multerer⁹, Ross E Myers¹⁰, Keith Ponitz¹¹, Adam Rosenberg¹², Jennifer B Soep¹³, Daniel C West¹⁴, Robert Englander¹⁵

Affiliations

¹ A. Schwartz is the Michael Reese Endowed Professor of Medical Education and research professor, pediatrics, University of Illinois College of Medicine, and network director, Association of Pediatric Program Directors (APPD) Longitudinal Educational Assessment Research Network (LEARN), Chicago, Illinois.
² D.F. Balmer is associate professor, pediatrics, The Children's Hospital of Pennsylvania and University of Pennsylvania, Philadelphia, Pennsylvania.
³ E. Borman-Shoap is vice chair of education, pediatric residency program director, and assistant professor, pediatrics, University of Minnesota, Minneapolis, Minnesota.
⁴ A. Chin is assistant clinical professor, pediatrics, University of California, Los Angeles, Mattel Children's Hospital, Los Angeles, California.
⁵ D. Henry is associate program director for assessment, and clinical competency committee chair, pediatric residency, University of California, San Francisco, San Francisco, California.
⁶ B.E. Herman is professor and vice chair of education and residency programs, pediatrics, University of Utah School of Medicine, Salt Lake City, Utah.
⁷ P. Hobday is assistant professor, pediatrics, University of Minnesota Medical School, Minneapolis, Minnesota.
⁸ J.H. Lee is associate director, pediatrics residency program, and assistant professor, pediatrics, the David Geffen School of Medicine at University of California, Los Angeles, Los Angeles, California.
⁹ S. Multerer is director, pediatric residency program, and associate professor, pediatric hospital medicine, University of Louisville Department of Pediatrics and Norton Children's Hospital, Louisville, Kentucky.
¹⁰ R.E. Myers is associate director, pediatric residency program, and associate professor, pediatrics, Rainbow Babies and Children's Hospital and Case Western Reserve University School of Medicine, Cleveland, Ohio.
¹¹ K. Ponitz is director, pediatric residency training program, pediatrics, Rainbow Babies and Children's Hospital, Cleveland, Ohio.
¹² A. Rosenberg is professor of pediatrics and program director, pediatric residency, University of Colorado School of Medicine, Children's Hospital Colorado, Aurora, Colorado.
¹³ J.B. Soep is associate professor, pediatrics, University of Colorado, Aurora, Colorado.
¹⁴ D.C. West is professor and associate chair for education, pediatrics, Children's Hospital of Philadelphia and Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania.
¹⁵ R. Englander is associate dean, undergraduate medical education, and professor, pediatrics, University of Minnesota Medical School, Minneapolis, Minnesota.

PMID: 32769469
DOI: 10.1097/ACM.0000000000003638

Comparative Study

Shared Mental Models Among Clinical Competency Committees in the Context of Time-Variable, Competency-Based Advancement to Residency

Alan Schwartz et al. Acad Med. 2020 Nov.

. 2020 Nov;95(11S Association of American Medical Colleges Learn Serve Lead: Proceedings of the 59th Annual Research in Medical Education Presentations):S95-S102.

doi: 10.1097/ACM.0000000000003638.

Authors

Affiliations

¹ A. Schwartz is the Michael Reese Endowed Professor of Medical Education and research professor, pediatrics, University of Illinois College of Medicine, and network director, Association of Pediatric Program Directors (APPD) Longitudinal Educational Assessment Research Network (LEARN), Chicago, Illinois.
² D.F. Balmer is associate professor, pediatrics, The Children's Hospital of Pennsylvania and University of Pennsylvania, Philadelphia, Pennsylvania.
³ E. Borman-Shoap is vice chair of education, pediatric residency program director, and assistant professor, pediatrics, University of Minnesota, Minneapolis, Minnesota.
⁴ A. Chin is assistant clinical professor, pediatrics, University of California, Los Angeles, Mattel Children's Hospital, Los Angeles, California.
⁵ D. Henry is associate program director for assessment, and clinical competency committee chair, pediatric residency, University of California, San Francisco, San Francisco, California.
⁶ B.E. Herman is professor and vice chair of education and residency programs, pediatrics, University of Utah School of Medicine, Salt Lake City, Utah.
⁷ P. Hobday is assistant professor, pediatrics, University of Minnesota Medical School, Minneapolis, Minnesota.
⁸ J.H. Lee is associate director, pediatrics residency program, and assistant professor, pediatrics, the David Geffen School of Medicine at University of California, Los Angeles, Los Angeles, California.
⁹ S. Multerer is director, pediatric residency program, and associate professor, pediatric hospital medicine, University of Louisville Department of Pediatrics and Norton Children's Hospital, Louisville, Kentucky.
¹⁰ R.E. Myers is associate director, pediatric residency program, and associate professor, pediatrics, Rainbow Babies and Children's Hospital and Case Western Reserve University School of Medicine, Cleveland, Ohio.
¹¹ K. Ponitz is director, pediatric residency training program, pediatrics, Rainbow Babies and Children's Hospital, Cleveland, Ohio.
¹² A. Rosenberg is professor of pediatrics and program director, pediatric residency, University of Colorado School of Medicine, Children's Hospital Colorado, Aurora, Colorado.
¹³ J.B. Soep is associate professor, pediatrics, University of Colorado, Aurora, Colorado.
¹⁴ D.C. West is professor and associate chair for education, pediatrics, Children's Hospital of Philadelphia and Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania.
¹⁵ R. Englander is associate dean, undergraduate medical education, and professor, pediatrics, University of Minnesota Medical School, Minneapolis, Minnesota.

PMID: 32769469
DOI: 10.1097/ACM.0000000000003638

Abstract

Purpose: To evaluate response process validity evidence for clinical competency committee (CCC) assessments of first-year residents on a subset of General Pediatrics Entrustable Professional Activities (EPAs) and milestones in the context of a national pilot of competency-based, time-variable (CBTV) advancement from undergraduate to graduate medical education.

Method: Assessments of 2 EPAs and 8 milestones made by the trainees' actual CCCs and 2 different blinded "virtual" CCCs for 48 first-year pediatrics residents at 4 residency programs between 2016 and 2018 were compared. Residents had 3 different training paths from medical school to residency: time-variable graduation at the same institution as their residency, time-fixed graduation at the same institution, or time-fixed graduation from a different institution. Assessments were compared using ordinal mixed-effects models.

Results: Actual CCCs assigned residents higher scores than virtual CCCs on milestones and one EPA's supervision levels. Residents who graduated from a different institution than their residency received lower milestone ratings than either group from the same institution; CBTV residents received higher ratings on one milestone (ICS4) and similar ratings on all others compared with non-CBTV residents who completed medical school at the same institution.

Conclusions: First-year residents who graduated from CBTV medical school programs were assessed as having the same level of competence as residents who graduated from traditional medical school programs, but response process evidence suggests that members of CCCs may also draw on undocumented personal knowledge of the learner to draw conclusions about resident competence.

PubMed Disclaimer

References

1. Frank JR, Snell LS, Cate OT, et al. Competency-based medical education: Theory to practice. Med Teach. 2010; 32:638–645
1. Powell DE, Carraccio C, Aschenbrener CA. Pediatrics redesign project: A pilot implementing competency-based education across the continuum. Acad Med. 2011; 86:e13
1. Andrews JS, Bale JF Jr, Soep JB, et al.; EPAC Study Group. Education in Pediatrics Across the Continuum (EPAC): First steps toward realizing the dream of competency-based education. Acad Med. 2018; 93:414–420
1. Murray KE, Lane JL, Carraccio C, et al.; Education in Pediatrics Across the Continuum (EPAC) Study Group. Crossing the gap: Using competency-based assessment to determine whether learners are ready for the undergraduate-to-graduate transition. Acad Med. 2019; 94:338–345
1. Obeso V, Brown D, Aiyer M, et al. Core Entrustable Professional Activities for Entering Residency: Toolkits for the 13 Core EPAs. Washington, DC: Association of American Medical Colleges; 2017. https://www.aamc.org/system/files/c/2/482214-epa13toolkit.pdf. Accessed August 11, 2020.

Publication types

Actions
Actions

MeSH terms

Actions
Actions
Actions
Actions
Actions

LinkOut - more resources

Research Materials
- NCI CPTC Antibody Characterization Program

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Shared Mental Models Among Clinical Competency Committees in the Context of Time-Variable, Competency-Based Advancement to Residency

Affiliations

Shared Mental Models Among Clinical Competency Committees in the Context of Time-Variable, Competency-Based Advancement to Residency

Authors

Affiliations

Abstract

References

Publication types

MeSH terms

LinkOut - more resources

Research Materials