From Model Organisms to Humans, the Opportunity for More Rigor in Methodologic and Statistical Analysis, Design, and Interpretation of Aging and Senescence Research

Abstract

This review identifies frequent design and analysis errors in aging and senescence research and discusses best practices in study design, statistical methods, analyses, and interpretation. Recommendations are offered for how to avoid these problems. The following issues are addressed: (a) errors in randomization, (b) errors related to testing within-group instead of between-group differences, (c) failing to account for clustering, (d) failing to consider interference effects, (e) standardizing metrics of effect size, (f) maximum life-span testing, (g) testing for effects beyond the mean, (h) tests for power and sample size, (i) compression of morbidity versus survival curve squaring, and (j) other hot topics, including modeling high-dimensional data and complex relationships and assessing model assumptions and biases. We hope that bringing increased awareness of these topics to the scientific community will emphasize the importance of employing sound statistical practices in all aspects of aging and senescence research.

Keywords: Geroscience; Methodologies; Reproducibility.

Figure 1.

Two examples of the differences in nominal significance (DINS) error. The DINS error occurs when the nominal significance (eg, if a result is called “statistically significant” by meeting a threshold like p < .05) of 2 different results is compared, rather than a direct between-group comparison. (A) The change in outcome during Treatment A is marked as statistically significantly greater than 0 (demarked with an asterisk), while the change during Treatment B is not statistically significantly different from 0. A DINS error would occur if authors concluded Treatment A worked better than Treatment B, when there is no compelling difference between the 2 groups, if they were compared directly. (B) The regression lines for 2 different groups (eg, different sexes) as a function of an exposure are nearly identical, with the dashed line having a slope significantly different from 0 and the solid line having a slope not significantly different from 0. If authors concluded these 2 regression lines were different based on statistical significance (eg, that there was a sex difference in the exposure–outcome relationship), they would be committing a DINS error, as the lines are not statistically different from each other if compared directly.