Biostatistics Series Module 6: Correlation and Linear Regression

Abstract

Correlation and linear regression are the most commonly used techniques for quantifying the association between two numeric variables. Correlation quantifies the strength of the linear relationship between paired variables, expressing this as a correlation coefficient. If both variables x and y are normally distributed, we calculate Pearson's correlation coefficient (r). If normality assumption is not met for one or both variables in a correlation analysis, a rank correlation coefficient, such as Spearman's rho (ρ) may be calculated. A hypothesis test of correlation tests whether the linear relationship between the two variables holds in the underlying population, in which case it returns a P < 0.05. A 95% confidence interval of the correlation coefficient can also be calculated for an idea of the correlation in the population. The value r² denotes the proportion of the variability of the dependent variable y that can be attributed to its linear relation with the independent variable x and is called the coefficient of determination. Linear regression is a technique that attempts to link two correlated variables x and y in the form of a mathematical equation (y = a + bx), such that given the value of one variable the other may be predicted. In general, the method of least squares is applied to obtain the equation of the regression line. Correlation and linear regression analysis are based on certain assumptions pertaining to the data sets. If these assumptions are not met, misleading conclusions may be drawn. The first assumption is that of linear relationship between the two variables. A scatter plot is essential before embarking on any correlation-regression analysis to show that this is indeed the case. Outliers or clustering within data sets can distort the correlation coefficient value. Finally, it is vital to remember that though strong correlation can be a pointer toward causation, the two are not synonymous.

Keywords: Bland–Altman plot; Pearson's r; Spearman's rho; correlation; correlation coefficient; intraclass correlation coefficient; method of least squares; point biserial correlation coefficient; regression.

Figure 1

Scatter diagram depicting direct and inverse linear relationships

Figure 2

Scatter diagram depicting a curvilinear relationship

Figure 3

Scatter diagram depicting relationship patterns between two variables

Figure 4

The principle of the method of least squares for linear regression. The sum of the squared “Residuals” is the least for the line of best fit

Figure 5

Examples of misleading correlations

Figure 6

Example of a Bland–Altman plot used to compare two test methods. The bias line with the limits of agreement is provided

Figure 7

Example of a Bland–Altman plot showing proportional bias. In this case, the difference between the methods first tends to narrow down and then increase as the value of measurements increase