Framework for the treatment and reporting of missing data in observational studies: The Treatment And Reporting of Missing data in Observational Studies framework

Abstract

Missing data are ubiquitous in medical research. Although there is increasing guidance on how to handle missing data, practice is changing slowly and misapprehensions abound, particularly in observational research. Importantly, the lack of transparency around methodological decisions is threatening the validity and reproducibility of modern research. We present a practical framework for handling and reporting the analysis of incomplete data in observational studies, which we illustrate using a case study from the Avon Longitudinal Study of Parents and Children. The framework consists of three steps: 1) Develop an analysis plan specifying the analysis model and how missing data are going to be addressed. An important consideration is whether a complete records' analysis is likely to be valid, whether multiple imputation or an alternative approach is likely to offer benefits and whether a sensitivity analysis regarding the missingness mechanism is required; 2) Examine the data, checking the methods outlined in the analysis plan are appropriate, and conduct the preplanned analysis; and 3) Report the results, including a description of the missing data, details on how the missing data were addressed, and the results from all analyses, interpreted in light of the missing data and the clinical relevance. This framework seeks to support researchers in thinking systematically about missing data and transparently reporting the potential effect on the study results, therefore increasing the confidence in and reproducibility of research findings.

Keywords: ALSPAC; Missing data; Multiple imputation; Observational studies; Reporting; STRATOS initiative.

Fig. 1

The framework.

Fig. 2

Causal diagram for the Avon Longitudinal Study of Parents and Children (ALSPAC) case study. Note, this figure illustrates the fact that we expect missingness to depend on the outcome of interest, educational attainment, as well as smoking itself, and that we expect there will be potential auxiliary variables that are both associated with missingness and with the incomplete exposure variable (smoking age 14 years).

Fig. 3

Flowchart for selecting an appropriate method to handle the missing data. ∗ The exception is if there is missingness in the exposure or covariates that is unrealted to the outcome but is missing not at random; in this context although inference from a complete records analysis would be unbiased, inference following multiple imputataion would be biased.