Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Multicenter Study
. 2025 Jan;18(1):e011159.
doi: 10.1161/CIRCOUTCOMES.124.011159. Epub 2025 Jan 3.

Evaluation of Peer Review of Percutaneous Coronary Intervention Operator Performance

Jacob A Doll  1   2   3 , Annika L Hebbe  3   4 , Carol E Simons  1 , Elliot J Stein  2 , Stephan Eisenbarth  3   4 , Stephen W Waldo  3   4   5 , Sunil V Rao  6 , David H Au  2
Affiliations
Multicenter Study

Evaluation of Peer Review of Percutaneous Coronary Intervention Operator Performance

Jacob A Doll et al. Circ Cardiovasc Qual Outcomes. 2025 Jan.

Abstract

Background: Case-based peer review of percutaneous coronary intervention (PCI) is used by many hospitals for quality improvement and to make decisions regarding physician competency. However, there are no studies testing the reliability or validity of peer review for PCI performance evaluation.

Methods: We recruited interventional cardiologists from 12 Veterans Affairs Health System facilities throughout the United States to provide PCI cases for review. Ten reviewers performed blinded reviews such that each case was reviewed twice. Cases were rated on a scale of 1 to 5 (with 5 being the best) for 6 care domains (Appropriateness, Lesion Suitability, Strategy, Technical Performance, Outcome, and Documentation) with a summary performance score calculated as the average of all domains. Separately, reviewers determined whether the standard of care was met. Interobserver reliability of the summary performance score was calculated using interclass correlation coefficient. We examined procedural complications and 30-day mortality and major adverse cardiac events for all PCIs performed by these operators from 2019 to 2022 when stratified in tertiles by summary performance score.

Results: Of the 65 cases provided by 13 operators, the mean summary performance score was 3.90 (SD=0.78) out of 5. The interclass correlation coefficient was 0.53, indicating moderate interobserver reliability. For 19 cases (29.2%), 1 reviewer indicated that the performance did not meet the standard of care; however, the second reviewer disagreed in all these cases. Average performance scores ranged from 3.35 to 4.38. Among the 3390 PCIs performed by reviewed cardiologists from 2019 to 2022, the lowest-rated tertile had higher rates of complications (2.9% versus 1.8%, P<0.01) and major adverse cardiac events (10.6% versus 8.0%, P<0.01) compared with the highest-rated tertile.

Conclusions: Case-based peer review identifies variation in physician performance that is correlated with PCI outcomes. However, reviewer disagreements about the standard of care raise concerns about the use of peer review for high-stakes assessments of physician competency.

Keywords: credentialing; peer review; percutaneous coronary intervention; quality improvement; standard of care.

PubMed Disclaimer

Conflict of interest statement

Dr Waldo receives investigator-initiated research support to the Denver Research Institute from Abiomed, Cardiovascular Systems Inc., Janssen Pharmaceuticals, and Merck Pharmaceuticals. He serves as the national director and Dr Doll serves as the assistant director for the Clinical Assessment, Reporting, and Tracking Program. The other authors report no conflicts.

Figures

Figure 1.
Figure 1.
Results of the generalizability theory analysis indicating the required number of cases and reviewers to achieve evaluation reliability that is acceptable (generalizability coefficient ≥0.7) or excellent (generalizability coefficient ≥0.9).
Figure 2.
Figure 2.
Outcomes of 3390 patients treated by percutaneous coronary intervention operators stratified in tertiles by peer review summary performance score, 2019 to 2022. Procedural complications include death, stroke, emergent coronary artery bypass graft surgery, major coronary dissection, coronary perforation, tamponade, access site injury requiring surgical intervention, and acute cardiogenic shock. Major adverse cardiac events (MACE) includes death, myocardial infarction, and coronary revascularization.
See this image and copyright information in PMC

Comment in

References

    1. Fanaroff AC, Zakroysky P, Dai D, Wojdyla D, Sherwood MW, Roe MT, Wang TY, Peterson ED, Gurm HS, Cohen MG, et al. Outcomes of PCI in relation to procedural characteristics and operator volumes in the United States. J Am Coll Cardiol. 2017;69:2913–2924. doi: 10.1016/j.jacc.2017.04.032 - PMC - PubMed
    1. Doll JA, Dai D, Roe MT, Messenger JC, Sherwood MW, Prasad A, Mahmud E, Rumsfeld JS, Wang TY, Peterson ED, et al. Assessment of operator variability in risk-standardized mortality following percutaneous coronary intervention: a report from the NCDR. JACC Cardiovasc Interv. 2017;10:672–682. doi: 10.1016/j.jcin.2016.12.019 - PubMed
    1. Doll JA, Nelson AJ, Kaltenbach LA, Wojdyla D, Waldo SW, Rao SV, Wang TY. Percutaneous coronary intervention operator profiles and associations with in-hospital mortality. Circ Cardiovasc Interv. 2022;15:e010909. doi: 10.1161/CIRCINTERVENTIONS.121.010909 - PubMed
    1. Harold JG, Bass TA, Bashore TM, Brindis RG, Brush JE, Jr, Burke JA, Dehmer GJ, Deychak YA, Jneid H, Jollis JG, et al. ACCF/AHA/SCAI 2013 update of the clinical competence statement on coronary artery interventional procedures: a report of the American College of Cardiology Foundation/American Heart Association/American College of Physicians Task Force on Clinical Competence and Training (Writing Committee to Revise the 2007 Clinical Competence Statement on Cardiac Interventional Procedures). J Am Coll Cardiol. 2013;62:357–396. doi: 10.1016/j.jacc.2013.05.002 - PubMed
    1. Edwards MT. The objective impact of clinical peer review on hospital quality and safety. Am J Med Qual. 2011;26:110–119. doi: 10.1177/1062860610380732 - PubMed

Publication types

MeSH terms

LinkOut - more resources