The transitive fallacy for randomized trials: if A bests B and B bests C in separate trials, is A better than C?

Abstract

Background: If intervention A bests B in one randomized trial, and B bests C in another randomized trial, can one conclude that A is better than C? The problem was motivated by the planning of a randomized trial, where A is spiral-CT screening, B is x-ray screening, and C is no screening. On its surface, this would appear to be a straightforward application of the transitive principle of logic.

Methods: We extended the graphical approach for omitted binary variables that was originally developed to illustrate Simpson's paradox, applying it to hypothetical, but plausible scenarios involving lung cancer screening, treatment for gastric cancer, and antibiotic therapy for clinical pneumonia.

Results: Graphical illustrations of the three examples show different ways the transitive fallacy for randomized trials can arise due to changes in an unobserved or unadjusted binary variable. In the most dramatic scenario, B bests C in the first trial, A bests B in the second trial, but C bests A at the time of the second trial.

Conclusion: Even with large sample sizes, combining results from a previous randomized trial of B versus C with results from a new randomized trial of A versus B will not guarantee correct inference about A versus C. A three-arm trial of A, B, and C would protect against this problem and should be considered when the sequential trials are performed in the context of changing secular trends in important omitted variables such as therapy in cancer screening trials.

Figure 1

Screening for lung cancer. Hypothetical results are shown for A = spiral CT, B = x-ray, and C = no screening. In the sequential study, results for B and C are similar in the first trial (when 30% of the subjects receive the new therapy), and results for A and B are similar in the second trial (when 70% of the subjects receive the new therapy). However, as shown with three-arm trial, it is incorrect to make the transitive inference that when 70% of the subjects receive the new therapy, the results for A and C would be similar.

Figure 2

Treatment for gastric cancer. Hypothetical results are shown for A = radical gastrectomy / splenectomy, B = "simple" gastrectomy, and C = radiation. In the sequential study, results for B and C differ in the first trial (when 30% of the subjects receive effective supportive care), and results for A and B are similar in the second trial (when 70% of the subjects receive effective supportive care). However, as shown with three-arm trial, it is incorrect to make the transitive inference that, when 70% of the subjects receive effective supportive care, the results for A and C would differ.

Figure 3

Antibiotic treatment for clinical pneumonia. Hypothetical results are shown for A = antibiotic for gram-positive, B = antibiotic for gram-negative, C = antibiotic for gram-positive that is more effective than A. In the sequential study, A bests B in the first trial (when 80% of the subjects are gram positive), and B bests C in the second trial (when 10% of the subjects are gram positive). However, as shown with the three-arm trial, it would be incorrect to make the transitive inference that when 10% of the subjects are gram-positive, A is better than C.