Statistical Inferences Using Effect Sizes in Human Endothelial Function Research

Abstract

Introduction: Magnitudes of change in endothelial function research can be articulated using effect size statistics. Effect sizes are commonly used in reference to Cohen's seminal guidelines of small (d = 0.2), medium (d = 0.5), and large (d = 0.8). Quantitative analyses of effect size distributions across various research disciplines have revealed values differing from Cohen's original recommendations. Here we examine effect size distributions in human endothelial function research, and the magnitude of small, medium, and large effects for macro and microvascular endothelial function.

Methods: Effect sizes reported as standardized mean differences were extracted from meta research available for endothelial function. A frequency distribution was constructed to sort effect sizes. The 25th, 50th, and 75th percentiles were used to derive small, medium, and large effects. Group sample sizes and publication year from primary studies were also extracted to observe any potential trends, related to these factors, in effect size reporting in endothelial function research.

Results: Seven hundred fifty-two effect sizes were extracted from eligible meta-analyses. We determined small (d = 0.28), medium (d = 0.69), and large (d = 1.21) effects for endothelial function that corresponded to the 25th, 50th, and 75th percentile of the data distribution.

Conclusion: Our data indicate that direct application of Cohen's guidelines would underestimate the magnitude of effects in human endothelial function research. This investigation facilitates future a priori power analyses, provides a practical guiding benchmark for the contextualization of an effect when no other information is available, and further encourages the reporting of effect sizes in endothelial function research.

Keywords: Effect sizes; Endothelial function; Statistical power; Statistics.

Fig. 1

Increased interest in effect sizes in scientific research over the last two decades as evidenced by articles containing the keyword ‘effect size’ in PubMed. Figure created using the ‘EuropePMC’ package [41]

Fig. 2

Flowchart depicting meta-analysis inclusion based on a PubMed search query entered April 2021. SMD standardized mean difference

Fig. 3

Histogram depicting the effect size distribution for endothelial function research. Vertical red lines at the 25th, 50th, and 75th percentiles correspond to small (d = 0.28), medium (d = 0.69), and large (d = 1.21) Cohen’s d effect sizes

Fig. 4

Panel A depicts a density plot of the effect size distributions for macrovascular (n = 544; skewness = 2.61; kurtosis = 8.27) and microvascular (n = 205; skewness = 2.99; kurtosis = 9.85) endothelial function. Panel B depicts a density plot of the effect size distributions for different measures used to assess endothelial function such as ultrasonography (n = 559, skewness = 2.64, kurtosis = 8.53), plethysmography (n = 98, skewness = 3.23, kurtosis = 12.08), laser doppler (n = 60, skewness = 2.85, kurtosis = 8.39), and other measures (n = 29, skewness = 1.67, kurtosis = 1.74). Cohen’s d values greater than 4.0 were not shown for visualization purposes

Fig. 5

Correlates of effect sizes in endothelial function research. Panel A shows no relationship between Cohen’s d and the year of primary study publication. The vertical red lines indicate the years in which the guidelines for the assessment of macrovascular endothelial function were published. Panel B depicts a weak negative relationship between Cohen’s d and the logarithm of sample size

Fig. 6

Relationship between sample size (n = number of pairs) and effect size for a paired samples t test (Panel A), and sample size (n = number in each group) and effect size for independent samples t test (Panel B), across three different levels of statistical power. Fewer participants per group are required for a given effect size using the paired samples t test. Figures assume a two-tailed analysis, with α = 0.05. Adapted from Yu and Yagle [42]

Fig. 7

The statistical power of the observed summary effect size in each of the included meta-analyses, and the median statistical power for included meta-analyses displayed across a range of plausible true effect sizes (from δ = 0.1 to δ = 1)