Marital status and risk of cardiovascular disease - a multi-analyst study in epidemiology

Abstract

In multi-analyst studies, several analysts use the same data to independently investigate identical research questions. Multi-analyst studies have been conducted mainly in psychology, social sciences, and neuroscience, but rarely in epidemiology. Sixteen analyst groups (24 researchers) with backgrounds mainly in statistics, mathematics, and epidemiology were asked to independently perform an analysis on the influence of marital status (never married versus cohabiting married) on cardiovascular outcomes. They were asked to use data from the Survey of Health, Ageing and Retirement in Europe (SHARE), a panel study of 140,000 persons aged 50 years and above from 28 European countries and Israel, and to provide an effect estimate, a comment on their results, and the full syntax of their analyses. In additional analyses beyond the multi-analyst approach, one group selected an exemplary regression model and varied definitions of exposure and outcome and the confounder adjustment set. Each analysis was unique. The size of the 16 datasets used for the analyses ranged from 15,592 to 336,914 observations. The effect estimates (odds ratios, hazard ratios, or relative risks) ranged from 0.72 to 1.02 (reference: cohabiting married) in strictly or partly cross-sectional analyses and from 0.95 to 1.31 in strictly longitudinal analyses. The choice of regression models, adjustment sets for confounding, and variations in the precise definition of exposure and outcome, all had only small effects on the effect estimates. The range of results was mainly due to differences from cross-sectional versus longitudinal analyses rather than to single analytical decisions each of which had less influence.

Keywords: Arbitrary choices; Many analysts; Multi-analyst study; Multiverse analysis; Researcher degrees of freedom.

Fig. 1

Effects estimates for the association between marital status (never married versus cohabiting married) and cardiovascular disease from the main analyses of the 16 analysts groups

Fig. 2

Effect estimates for the association between marital status (never married versus cohabiting married) and cardiovascular disease only from strictly longitudinal analyses

Fig. 3

Effect estimates for the association between marital status (never married versus cohabiting married) and cardiovascular disease from strictly or partly cross-sectional analyses

Fig. 4

Relative risks with 95% confidence interval (CI) of a longitudinal analysis (baseline = wave 1) with all adjustment sets used by the 16 analysts groups (for adjustments sets, cf. Table 1)