Completeness of Follow-Up Determines Validity of Study Findings: Results of a Prospective Repeated Measures Cohort Study

Abstract

Background: Current reporting guidelines do not call for standardised declaration of follow-up completeness, although study validity depends on the representativeness of measured outcomes. The Follow-Up Index (FUI) describes follow-up completeness at a given study end date as ratio between the investigated and the potential follow-up period. The association between FUI and the accuracy of survival-estimates was investigated.

Methods: FUI and Kaplan-Meier estimates were calculated twice for 1207 consecutive patients undergoing aortic repair during an 11-year period: in a scenario A the population's clinical routine follow-up data (available from a prospective registry) was analysed conventionally. For the control scenario B, an independent survey was completed at the predefined study end. To determine the relation between FUI and the accuracy of study findings, discrepancies between scenarios regarding FUI, follow-up duration and cumulative survival-estimates were evaluated using multivariate analyses.

Results: Scenario A noted 89 deaths (7.4%) during a mean considered follow-up of 30±28months. Scenario B, although analysing the same study period, detected 304 deaths (25.2%, P<0.001) as it scrutinized the complete follow-up period (49±32months). FUI (0.57±0.35 versus 1.00±0, P<0.001) and cumulative survival estimates (78.7% versus 50.7%, P<0.001) differed significantly between scenarios, suggesting that incomplete follow-up information led to underestimation of mortality. Degree of follow-up completeness (i.e. FUI-quartiles and FUI-intervals) correlated directly with accuracy of study findings: underestimation of long-term mortality increased almost linearly by 30% with every 0.1 drop in FUI (adjusted HR 1.30; 95%-CI 1.24;1.36, P<0.001).

Conclusion: Follow-up completeness is a pre-requisite for reliable outcome assessment and should be declared systematically. FUI represents a simple measure suited as reporting standard. Evidence lacking such information must be challenged as potentially flawed by selection bias.

Fig 1. Patient flow through the study.

Interventions were identified from a prospective registry of consecutive aortic interventions. Each patient was included only once (i.e., for the latest aortic intervention) during the study period.

Fig 2. Proposed principle of follow-up assessment.

Individual follow-up is characterized by two indicators: absolute duration and completeness. The duration measures the time, for which valid information on the investigated outcome is available (patients 1 to 6), but must end at the study closing date, even if information becomes available thereafter (patients 2 and 6). Similarly, clinical outcome is defined at this very closing date (patient 6). Summary statistics exclude those known to have died (patient 3) as well as those lost to follow-up within 30 days (patient 7). Both subgroups are reported separately as proportions; those who have died with a median time to death. The completeness, in contrast, is expressed as proportion (follow-up index, FUI), calculated as displayed. Patients known to be alive (patients 1 and 2) and patients known to be dead (patient 3) carry a FUI of 1 by default, all others have a FUI between 0 and 1. The unaccounted follow-up period (1 minus FUI) may hide events (patient 5) leading to underestimation bias. Therefore, the closer the FUI to 1 the smaller the risk of selection bias.

Fig 3. Kaplan Meier long-term survival estimates for the study population (n = 1207) according to completeness of follow-up.

Scenario A (blue curve) estimated survival based on registry data, which, although collected prospectively during clinical routine, were not up to date for every patient at the study end. Scenario B (red curve), however, estimated survival of the same study population based on a comprehensive survey performed at the study end. Completeness of follow-up differed significantly between scenarios as expressed as follow-up index (FUI, see text). Thereby, scenario A (FUI 0.57±0.35) underestimated effective mortality by almost 30% (scenario B; FUI 1.0±0).

Fig 4. Association between follow-up index (FUI) and the degree of underestimated mortality among ‘potential survivors’ (n = 1116).

Patients were grouped into equally sized quartiles according to FUI (quartile 1 with highest FUIs; quartile 4 with lowest FUIs). After adjustment for potential confounding factors, underestimation of the actual mortality (i.e., inaccuracy of outcome estimate) correlated significantly with decreasing completeness of follow-up (see Table 2).