Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Jun 1;181(6):747-755.
doi: 10.1001/jamainternmed.2021.0269.

Accuracy of Practitioner Estimates of Probability of Diagnosis Before and After Testing

Daniel J Morgan  1   2 , Lisa Pineles  1 , Jill Owczarzak  3 , Larry Magder  1 , Laura Scherer  4   5   6 , Jessica P Brown  1 , Chris Pfeiffer  7 , Chris Terndrup  7 , Luci Leykum  8   9 , David Feldstein  10 , Andrew Foy  11   12 , Deborah Stevens  1 , Christina Koch  13 , Max Masnick  14 , Scott Weisenberg  15 , Deborah Korenstein  16
Affiliations

Accuracy of Practitioner Estimates of Probability of Diagnosis Before and After Testing

Daniel J Morgan et al. JAMA Intern Med. .

Abstract

Importance: Accurate diagnosis is essential to proper patient care.

Objective: To explore practitioner understanding of diagnostic reasoning.

Design, setting, and participants: In this survey study, 723 practitioners at outpatient clinics in 8 US states were asked to estimate the probability of disease for 4 scenarios common in primary care (pneumonia, cardiac ischemia, breast cancer screening, and urinary tract infection) and the association of positive and negative test results with disease probability from June 1, 2018, to November 26, 2019. Of these practitioners, 585 responded to the survey, and 553 answered all of the questions. An expert panel developed the survey and determined correct responses based on literature review.

Results: A total of 553 (290 resident physicians, 202 attending physicians, and 61 nurse practitioners and physician assistants) of 723 practitioners (76.5%) fully completed the survey (median age, 32 years; interquartile range, 29-44 years; 293 female [53.0%]; 296 [53.5%] White). Pretest probability was overestimated in all scenarios. Probabilities of disease after positive results were overestimated as follows: pneumonia after positive radiology results, 95% (evidence range, 46%-65%; comparison P < .001); breast cancer after positive mammography results, 50% (evidence range, 3%-9%; P < .001); cardiac ischemia after positive stress test result, 70% (evidence range, 2%-11%; P < .001); and urinary tract infection after positive urine culture result, 80% (evidence range, 0%-8.3%; P < .001). Overestimates of probability of disease with negative results were also observed as follows: pneumonia after negative radiography results, 50% (evidence range, 10%-19%; P < .001); breast cancer after negative mammography results, 5% (evidence range, <0.05%; P < .001); cardiac ischemia after negative stress test result, 5% (evidence range, 0.43%-2.5%; P < .001); and urinary tract infection after negative urine culture result, 5% (evidence range, 0%-0.11%; P < .001). Probability adjustments in response to test results varied from accurate to overestimates of risk by type of test (imputed median positive and negative likelihood ratios [LRs] for practitioners for chest radiography for pneumonia: positive LR, 4.8; evidence, 2.6; negative LR, 0.3; evidence, 0.3; mammography for breast cancer: positive LR, 44.3; evidence range, 13.0-33.0; negative LR, 1.0; evidence range, 0.05-0.24; exercise stress test for cardiac ischemia: positive LR, 21.0; evidence range, 2.0-2.7; negative LR, 0.6; evidence range, 0.5-0.6; urine culture for urinary tract infection: positive LR, 9.0; evidence, 9.0; negative LR, 0.1; evidence, 0.1).

Conclusions and relevance: This survey study suggests that for common diseases and tests, practitioners overestimate the probability of disease before and after testing. Pretest probability was overestimated in all scenarios, whereas adjustment in probability after a positive or negative result varied by test. Widespread overestimates of the probability of disease likely contribute to overdiagnosis and overuse.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest Disclosures: Dr Morgan reported receiving grants from the National Institutes of Health (NIH) during the conduct of the study and grants from the US Department of Veterans Affairs, the Agency for Healthcare Research and Quality, and the Centers for Disease Control and Prevention outside the submitted work. Ms Pineles reported receiving grants from the NIH to the University of Maryland School of Medicine during the conduct of the study. Dr Scherer reported receiving grants from the NIH during the conduct of the study. Dr Brown reported receiving grants from the NIH during the conduct of the study. Dr Pfeiffer reported receiving grants from Pfizer to serve as site investigator for a Clostridium difficile vaccine trial (protocol B5091007) since July 2020 under a Cooperative Research and Development Agreement with VA Portland outside the submitted work. Dr Korenstein reported receiving grants from the NIH and grants from the National Cancer Institute to Memorial Sloan Kettering Cancer Center during the conduct of the study and that her spouse serves on the scientific advisory board and as a consultant for Vedanta Biosciences, serves as a consultant for Takeda, serves on the scientific advisory board and as a consultant for Opentrons. No other disclosures were reported.

Figures

Figure.
Figure.. Distribution of Practitioner Assessments of Probability of Disease Before Testing and After Positive or Negative Test Results for 4 Testing Questions Representing Scenarios Commonly Encountered in Primary Care
A, Scenario: a previously healthy 35-year-old woman who smokes tobacco presents with 5 days of fatigue, productive cough, worsening shortness of breath, temperatures to 38.9°C, and decreased breath sounds in the lower right field. She has a heart rate of 105 beats/min, but vital signs are otherwise normal. B, Scenario: a 45-year-old woman comes in for an annual visit. She has no specific risk factors or symptoms for breast cancer. C, Scenario: a 43-year-old premenopausal woman presents with atypical chest pain and normal ECG results. She has no risk factors and has normal vital signs and examination findings. D, Scenario: a 65-year-old man is seen for osteoarthritis. He has noted foul-smelling urine and no pain or difficulty with urination. A urine dipstick shows trace blood. ECG indicates electrocardiography.
See this image and copyright information in PMC

Comment in

References

    1. Hunter DJ. Uncertainty in the era of precision medicine. N Engl J Med. 2016;375(8):711-713. doi:10.1056/NEJMp1608282 - DOI - PubMed
    1. Schünemann HJ, Mustafa RA, Brozek J, et al. ; GRADE Working Group . GRADE guidelines: 22. the GRADE approach for tests and strategies—from test accuracy to patient-important outcomes and recommendations. J Clin Epidemiol. 2019;111:69-82. doi:10.1016/j.jclinepi.2019.02.003 - DOI - PubMed
    1. Armstrong KA, Metlay JP. Annals clinical decision making: using a diagnostic test. Ann Intern Med. 2020;172(9):604-609. doi:10.7326/M19-1940 - DOI - PubMed
    1. Centers for Disease Control and Prevention. About DLS. Published March 26, 2020. Accessed May 20, 2020. https://www.cdc.gov/csels/dls/about-us.html
    1. Whiting PF, Davenport C, Jameson C, et al. . How well do health professionals interpret diagnostic information? a systematic review. BMJ Open. 2015;5(7):e008155. doi:10.1136/bmjopen-2015-008155 - DOI - PMC - PubMed

Publication types