Estimates of sensitivity and specificity can be biased when reporting the results of the second test in a screening trial conducted in series

Abstract

Background: Cancer screening reduces cancer mortality when early detection allows successful treatment of otherwise fatal disease. There are a variety of trial designs used to find the best screening test. In a series screening trial design, the decision to conduct the second test is based on the results of the first test. Thus, the estimates of diagnostic accuracy for the second test are conditional, and may differ from unconditional estimates. The problem is further complicated when some cases are misclassified as non-cases due to incomplete disease status ascertainment.

Methods: For a series design, we assume that the second screening test is conducted only if the first test had negative results. We derive formulae for the conditional sensitivity and specificity of the second test in the presence of differential verification bias. For comparison, we also derive formulae for the sensitivity and specificity for a single test design, both with and without differential verification bias.

Results: Both the series design and differential verification bias have strong effects on estimates of sensitivity and specificity. In both the single test and series designs, differential verification bias inflates estimates of sensitivity and specificity. In general, for the series design, the inflation is smaller than that observed for a single test design.The degree of bias depends on disease prevalence, the proportion of misclassified cases, and on the correlation between the test results for cases. As disease prevalence increases, the observed conditional sensitivity is unaffected. However, there is an increasing upward bias in observed conditional specificity. As the proportion of correctly classified cases increases, the upward bias in observed conditional sensitivity and specificity decreases. As the agreement between the two screening tests becomes stronger, the upward bias in observed conditional sensitivity decreases, while the specificity bias increases.

Conclusions: In a series design, estimates of sensitivity and specificity for the second test are conditional estimates. These estimates must always be described in context of the design of the trial, and the study population, to prevent misleading comparisons. In addition, these estimates may be biased by incomplete disease status ascertainment.

Figure 1

Flowchart for single test design. Flowchart depicts a single test screening trial from an omniscient point of view. Dashed lines indicate a pathway that is unavailable to that class of participants (true case or true non-case) due to the assumptions of our model. The gray box indicates cases that are misclassified as noncases by the study investigator.

Figure 2

Flowchart for test if negative series design. Flowchart depicts a test if negative series screening trial from an omniscient point of view. Dashed lines indicate a pathway that is unavailable to that class of participants (true case or true non-case) due to the assumptions of our model. In A, non-cases who screen positive on Test 1 are given a reference test. The results of this test are negative. The study investigator then goes on to screen the participant with Test 2, in case the reference test has failed. In B, cases who screen positive on Test 1 are given a reference test. The results of this reference test are positive and the study participant is observed to have disease. The gray box indicates cases that are misclassified as non-cases by the study investigator. The design is similar to that of Lehman et al. [5].

Figure 3

Effect of disease prevalence on percent bias. Effect of disease prevalence on percent bias in observed sensitivity (A) and specificity (B). Parameter definitions are as in "Parameters" (Table 6), except that the disease prevalence is allowed to vary. Percent bias is the bias in observed sensitivity or specificity divided by the true sensitivity or specificity. The observed results for Test 2 in a test if negative series design are denoted by "Test 2 Series". The observed results for a single test design are denoted by "Single". The observed sensitivity is biased upwards by 14% for the single test design and 12% for the series design.

Figure 4

Effect of proportion elective procedure on percent bias. Effect of the proportion of participants who undergo an elective procedure on percent bias in observed sensitivity (A) and specificity (B). Note that the scale of the y-axis of the specificity graph (B) is enlarged to show minute changes. Parameter definitions are as in "Parameters" (Table 6), except that the proportion elective procedure is allowed to vary. Otherwise as Figure 3.

Figure 5

Effect of proportion of double negative cases on percent bias. Effect of the proportion of double negative cases on percent bias in observed sensitivity (A) and specificity (B). Note that the scale of the y-axis of the specificity graph (B) is enlarged to show minute changes. Parameter definitions are as in "Parameters" (Table 6), except that the proportion double negative cases is allowed to vary. Otherwise as Figure 3.