Risky business: Correlation and causation in longitudinal studies of skill development

Abstract

Developmental theories often posit that changes in children's early psychological characteristics will affect much later psychological, social, and economic outcomes. However, tests of these theories frequently yield results that are consistent with plausible alternative theories that posit a much smaller causal role for earlier levels of these psychological characteristics. Our article explores this issue with empirical tests of skill-building theories, which predict that early boosts to simpler skills (e.g., numeracy or literacy) or behaviors (e.g., antisocial behavior or executive functions) support the long-term development of more sophisticated skills or behaviors. Substantial longitudinal associations between academic or socioemotional skills measured early and then later in childhood or adolescence are often taken as support of these skill-building processes. Using the example of skill-building in mathematics, we argue that longitudinal correlations, even if adjusted for an extensive set of baseline covariates, constitute an insufficiently risky test of skill-building theories. We first show that experimental manipulation of early math skills generates much smaller effects on later math achievement than the nonexperimental literature has suggested. We then conduct falsification tests that show puzzlingly high cross-domain associations between early math and later literacy achievement. Finally, we show that a skill-building model positing a combination of unmeasured stable factors and skill-building processes can reproduce the pattern of experimental impacts on children's mathematics achievement. Implications for developmental theories, methods, and practice are discussed. (PsycINFO Database Record

Figure 1. Bivariate correlations with Fall of Kindergarten measures

Source: ECLS-K 1998–1999 cohort. All correlations are p<.05

Figure 2. Regression-adjusted correlations and experimental impacts in TRIAD

Note: All 4^th and 5^th grade impacts are p>.05. All correlations and other impacts are p<.05. Impacts are rescaled to be 1.0 in the spring of pre-K, Right scale shows non-rescaled impacts. Vertical lines depict 95% confidence intervals.

Figure 3

Direct and indirect paths in a math skill-building model

Figure 4. An alternative math skill-building model with unmeasured persistent influences

Predicted standardized treatment effects on math skill following 1 SD boost in Math skill at time 1: Math skill at time 1:1 Math skill at time 2:MS₁ Math skill at time 3: MS₁* MS₂

Figure 5. Correlations inferred from MS path estimates in Table 1

Note: All 4^th and 5^th grade impacts are p>.05. All correlations and other impacts are p<.05. Impacts are rescaled to be 1.0 in the spring of pre-K, Right scale shows non-rescaled impacts. Vertical lines depict 95% confidence intervals.