Assessing strength of evidence in diagnostic tests

Abstract

Clinical encounters between clinicians and patients begin with an attempt at diagnosis, a foundational element in determining a patient's ultimate outcome. Diagnosis that is expedient and accurate will result in a treatment that is expedient, appropriate, and cost-effective. In essence, evidence-based diagnosis is as vital as evidence-based intervention and treatment. If surgeons are committed to making expedient and accurate diagnoses, they must strive to apply diagnostic tests not just on the basis of ease, novelty, or availability but for the soundness of evidence behind them. In the scopes of both aesthetic and reconstructive surgery, advocating evidence-driven diagnostic test use is relevant. A pertinent example of how this relates to plastic surgery is the U.S. Food and Drug Administration recommendation to screen asymptomatic women with silicone breast implants with magnetic resonance imaging. For an important recommendation such as this that has tremendous cost implications to patients, sound study design and rigorous evaluation of the accuracy of magnetic resonance imaging as a screening tool has important health policy implications. The authors demonstrate how to determine the accuracy of diagnostic tests and, more importantly, illustrate the essential qualities of any study to establish the accuracy of a diagnostic test.

Figure 1

Figure 1a CTS(−): carpal tunnel syndrome absent CTS(+): carpal tunnel syndrome present This schematic shows a test with sensitivity and specificity of 100% respectively. There are no false positive or false negatives. There is a threshold value that perfectly delineates patients with CTS from patients without. This is akin to an ideal reference standard and a perfectly accurate diagnostic test. Figure 1b CTS (−): carpal tunnel syndrome is absent CTS (+): carpal tunnel syndrome is present This shows a test with values that overlap within a certain range for patients who are CTS (+) and CTS (−). This is imperfect when compared to the test in 1a. Sensitivity and specificity do not equal 1, the test is not perfectly accurate. The width of the overlap area (false positives and false negatives) is inversely proportional to its accuracy. The narrower the overlap area, the easier it is to find a threshold that can approximate the performance of the reference standard (better accuracy). The converse is true for wider areas of overlap. Figure 1c CTS (−): carpal tunnel syndrome is absent CTS (+): carpal tunnel syndrome is present Everyone to the right of the threshold is CTS (+). If the threshold is moved to the right, we eliminate CTS (−) patients (decrease false positives; increase specificity) but we also miss a substantial number of CTS (+) patients (increase false negatives: decrease sensitivity) If the area of overlap is narrow, the percentage of CTS (−) patients we had in the overlap region was small to begin with and the percentage of CTS (+) patients we miss by moving the threshold right will be small hence, the accuracy of the test will be close to that demonstrated in fig. 1a. If the overlap area is wide, we would have to move the threshold farther right to eliminate most false positives. And as you can see we would also have to sacrifice a larger percentage of our true positives. Compared to fig. 1a, the accuracy would be significantly inferior. Figure 1d CTS (−): carpal tunnel syndrome is absent CTS (+): carpal tunnel syndrome is present Every one to the right of the threshold is CTS (+). If the threshold is moved to the left, we capture most CTS (+) patients (decrease false negatives; increase sensitivity) but we also capture a substantial number of CTS (−) patients (increase false positives: decrease specificity). The width of overlap area applies here as well. As you can see, the width of the overlap area gives us its sensitivity and specificity values relative to the reference standard.

Figure 1

Figure 1a CTS(−): carpal tunnel syndrome absent CTS(+): carpal tunnel syndrome present This schematic shows a test with sensitivity and specificity of 100% respectively. There are no false positive or false negatives. There is a threshold value that perfectly delineates patients with CTS from patients without. This is akin to an ideal reference standard and a perfectly accurate diagnostic test. Figure 1b CTS (−): carpal tunnel syndrome is absent CTS (+): carpal tunnel syndrome is present This shows a test with values that overlap within a certain range for patients who are CTS (+) and CTS (−). This is imperfect when compared to the test in 1a. Sensitivity and specificity do not equal 1, the test is not perfectly accurate. The width of the overlap area (false positives and false negatives) is inversely proportional to its accuracy. The narrower the overlap area, the easier it is to find a threshold that can approximate the performance of the reference standard (better accuracy). The converse is true for wider areas of overlap. Figure 1c CTS (−): carpal tunnel syndrome is absent CTS (+): carpal tunnel syndrome is present Everyone to the right of the threshold is CTS (+). If the threshold is moved to the right, we eliminate CTS (−) patients (decrease false positives; increase specificity) but we also miss a substantial number of CTS (+) patients (increase false negatives: decrease sensitivity) If the area of overlap is narrow, the percentage of CTS (−) patients we had in the overlap region was small to begin with and the percentage of CTS (+) patients we miss by moving the threshold right will be small hence, the accuracy of the test will be close to that demonstrated in fig. 1a. If the overlap area is wide, we would have to move the threshold farther right to eliminate most false positives. And as you can see we would also have to sacrifice a larger percentage of our true positives. Compared to fig. 1a, the accuracy would be significantly inferior. Figure 1d CTS (−): carpal tunnel syndrome is absent CTS (+): carpal tunnel syndrome is present Every one to the right of the threshold is CTS (+). If the threshold is moved to the left, we capture most CTS (+) patients (decrease false negatives; increase sensitivity) but we also capture a substantial number of CTS (−) patients (increase false positives: decrease specificity). The width of overlap area applies here as well. As you can see, the width of the overlap area gives us its sensitivity and specificity values relative to the reference standard.

Figure 2

Formulas for the calculation of posttest probability from likelihood ratios.