Retrospective analyses of pivotal prospective studies with population segmentation: statistically based inferences and clinical relevance

Abstract

Retrospective analyses of pivotal prospective studies are important for verifying the inferences made as a result of the original studies and for generating new hypotheses. However, careful attention should be given to the comprehensiveness and completeness of a retrospective analysis and how it is ultimately used. A recent retrospective analysis of the Digital Mammographic Imaging Screening Trial (DMIST) underscores several important points related to inference generation and generalization of the results on the basis of summary performance indexes, as well as the importance of incorporating a clinically relevant perspective when generating inferences primarily on the basis of statistical test results. This article highlights three important points related to (1) the use of performance indexes (namely, area under the receiver-operating characteristic curve), (2) applied statistical methods (namely, Bonferroni corrections for multiple comparison), and (3) practical conclusions (namely, consideration of all possible inferences that could be generated from the data), as well as possible implications and limitations of these retrospective analyses. The discussion in this paper is based on one specific retrospective analysis of a prospective study, but the topics addressed are quite basic, general, and potentially applicable to a number of retrospective analyses of data that are experimentally ascertained during pivotal prospective studies, as well as during observer performance studies.