Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2009 Dec;1(2):327-33.
doi: 10.4300/JGME-D-09-00062.1.

The ACGME Resident Survey Aggregate Reports: An Analysis and Assessment of Overall Program Compliance

Kathleen D Holt, Rebecca S Miller

The ACGME Resident Survey Aggregate Reports: An Analysis and Assessment of Overall Program Compliance

Kathleen D Holt et al. J Grad Med Educ. 2009 Dec.

Abstract

Background: The Accreditation Council for Graduate Medical Education (ACGME) uses a 29-question Resident Survey for yearly residency program assessments. This article describes methodology for aggregating Resident Survey data into 5 discrete areas of program performance for use in the accreditation process. This article also describes methodology for setting thresholds that may assist Residency Review Committees in identifying programs with potential compliance problems.

Methods: A team of ACGME staff and Residency Review Committee chairpersons reviewed the survey for content and proposed thresholds (through a modified Angoff procedure) that would indicate problematic program functioning.

Results: Interrater agreement was high for the 5 content areas and for the threshold values (percentage of noncompliant residents), indicating that programs above these thresholds may warrant follow-up by the accrediting organization. Comparison of the Angoff procedure and the actual distribution of the data revealed that the Angoff thresholds were extremely similar to 1 standard deviation above the content area mean.

Conclusion: Data from the ACGME Resident Survey may be aggregated into internally consistent and consensually valid areas that may help Residency Review Committees make more targeted and specific judgments about program compliance.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Number of Programs Over Thresholds: Angoff Method Compared With Mean + 1 Standard Deviation
Figure 2
Figure 2
Number of Programs Above Mean + 1 Standard Deviation
Figure 3
Figure 3
Mean Noncompliance for Each Area, With Standard Deviation Shown
Figure 4
Figure 4
Distribution of Noncompliant Areas for Those Programs Having Only One Area Above a Threshold
Figure 5
Figure 5
Percentage of Programs Having 2, 3, 4, or 5 Areas Above the Thresholds
See this image and copyright information in PMC

References

    1. Holt K. D., Miller R. S., Philibert I., Heard J. K., Nasca T. J. Resident perspectives on the learning environment: data from the ACGME's resident survey. Acad. Med. In press. - PubMed
    1. Philibert I., Miller R., Heard J. K., Holt K. D. Assessing duty hour compliance: practical lessons for programs and institutions. J Grad Med Educ. 2009;1(1):166–167. - PMC - PubMed
    1. Angoff W. A. Educational Measurement. Washington, DC: American Council on Education; 1971.
    1. Fleiss J. L. Statistical Methods for Rates and Proportions. 2nd ed. New York: John Wiley; 1981.
    1. Landis J. R., Koch G. G. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174. - PubMed

LinkOut - more resources