Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Multicenter Study
. 2002 Sep 28;325(7366):679.
doi: 10.1136/bmj.325.7366.679.

Using standardised patients to measure physicians' practice: validation study using audio recordings

Jeff Luck  1 , John W Peabody
Affiliations
Multicenter Study

Using standardised patients to measure physicians' practice: validation study using audio recordings

Jeff Luck et al. BMJ. .

Abstract

Objective: To assess the validity of standardised patients to measure the quality of physicians' practice.

Design: Validation study of standardised patients' assessments. Physicians saw unannounced standardised patients presenting with common outpatient conditions. The standardised patients covertly tape recorded their visit and completed a checklist of quality criteria immediately afterwards. Their assessments were compared against independent assessments of the recordings by a trained medical records abstractor.

Setting: Four general internal medicine primary care clinics in California.

Participants: 144 randomly selected consenting physicians.

Main outcome measures: Rates of agreement between the patients' assessments and independent assessment.

Results: 40 visits, one per standardised patient, were recorded. The overall rate of agreement between the standardised patients' checklists and the independent assessment of the audio transcripts was 91% (kappa=0.81). Disaggregating the data by medical condition, site, level of physicians' training, and domain (stage of the consultation) gave similar rates of agreement. Sensitivity of the standardised patients' assessments was 95%, and specificity was 85%. The area under the receiver operator characteristic curve was 90%.

Conclusions: Standardised patients' assessments seem to be a valid measure of the quality of physicians' care for a variety of common medical conditions in actual outpatient settings. Properly trained standardised patients compare well with independent assessment of recordings of the consultations and may justify their use as a "gold standard" in comparing the quality of care across sites or evaluating data obtained from other sources, such as medical records and clinical vignettes.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Percentage of items on checklist done by physician, as rated by standardised patients and as indicated by audio recordings of visits
Figure 2
Figure 2
Receiver operator characteristic curve for standardised patients with respect to audio recordings
See this image and copyright information in PMC

Comment in

  • Communications and emotions.
    Buckman R. Buckman R. BMJ. 2002 Sep 28;325(7366):672. doi: 10.1136/bmj.325.7366.672. BMJ. 2002. PMID: 12351345 Free PMC article. No abstract available.

References

    1. Beullens J, Rethans JJ, Goedhuys J, Buntinx F. The use of standardized patients in research in general practice. Fam Pract. 1997;14:58–62. - PubMed
    1. Rethans JJ, Martin E, Metsemakers J. To what extent do clinical notes by general practitioners reflect actual medical practice? A study using simulated patients. Br J Gen Pract. 1994;44:153–156. - PMC - PubMed
    1. Kopelow ML, Schnabl GK, Hassard TH, Tamblyn RM, Klass DJ, Beazley G, et al. Assessing practicing physicians in two settings using standardized patients. Acad Med. 1992;67(10 suppl):S19–S21. - PubMed
    1. Woodward CA, McConvey GA, Neufeld V, Norman GR, Walsh A. Measurement of physician performance by standardized patients. Med Care. 1985;23:1019–1027. - PubMed
    1. Glassman PA, Luck J, O'Gara EM, Peabody JW. Using standardized patients to measure quality: evidence from the literature and a prospective study. Jt Comm J Qual Improv. 2000;26:644–653. - PubMed

Publication types