Mediating effect of pubertal stages on the family environment and neurodevelopment: An open-data replication and multiverse analysis of an ABCD Study®

Abstract

Increasing evidence demonstrates that environmental factors meaningfully impact the development of the brain (Hyde et al., 2020; McEwen and Akil, 2020). Recent work from the Adolescent Brain Cognitive Development (ABCD) Study^® suggests that puberty may indirectly account for some association between the family environment and brain structure and function (Thijssen et al., 2020). However, a limited number of large studies have evaluated what, how, and why environmental factors impact neurodevelopment. When these topics are investigated, there is typically inconsistent operationalization of variables between studies which may be measuring different aspects of the environment and thus different associations in the analytic models. Multiverse analyses (Steegen et al., 2016) are an efficacious technique for investigating the effect of different operationalizations of the same construct on underlying interpretations. While one of the assets of Thijssen et al. (2020) was its large sample from the ABCD data, the authors used an early release that contained 38% of the full ABCD sample. Then, the analyses used several 'researcher degrees of freedom' (Gelman and Loken, 2014) to operationalize key independent, mediating and dependent variables, including but not limited to, the use of a latent factor of preadolescents' environment comprised of different subfactors, such as parental monitoring and child-reported family conflict. While latent factors can improve reliability of constructs, the nuances of each subfactor and measure that comprise the environment may be lost, making the latent factors difficult to interpret in the context of individual differences. This study extends the work of Thijssen et al. (2020) by evaluating the extent to which the analytic choices in their study affected their conclusions. In Aim 1, using the same variables and models, we replicate findings from the original study using the full sample in Release 3.0. Then, in Aim 2, using a multiverse analysis we extend findings by considering nine alternative operationalizations of family environment, three of puberty, and five of brain measures (total of 135 models) to evaluate the impact on conclusions from Aim 1. In these results, 90% of the directions of effects and 60% of the p-values (e.g. p > .05 and p < .05) across effects were comparable between the two studies. However, raters agreed that only 60% of the effects had replicated. Across the multiverse analyses, there was a degree of variability in beta estimates across the environmental variables, and lack of consensus between parent reported and child reported pubertal development for the indirect effects. This study demonstrates the challenge in defining which effects replicate, the nuance across environmental variables in the ABCD data, and the lack of consensus across parent and child reported puberty scales in youth.

Keywords: Environment; Pubertal development; Resting state MRI; Structural MRI; Youth.

Fig. 1

Conceptual model for proposed analyses in Aim 1 and Aim 2. A: Mediation model from Thijssen et al. (2020) that is used in replication. B: Proposed models for the mediation analyses, varying across nine independent variables (IV) and three mediators.

Fig. 2

Correlations (r's) between several key variables and factors proposed in Aim 2. Blue-shaded boxes represent positive correlations; red-shaded boxes represent negative correlations; the darkness of the hue represents the magnitude of the correlations. FamEnv = Family Environment; Demo = Demographic; Fact = Factor; Par = Parent; Yth = Youth; FES = Family Environment Scale (i.e. Conflict; reverse scored); PMON = Parental Monitoring; Accept = Child Report of Parent Behavior Inventory (i.e., Acceptance); Avg IncEdu = Average Parent Reported Income & Education; PDS = Pubertal Development Scale; Amyg = Amygdala; ACC = Anterior Cingulate Cortex; CT = Cortical Thickness; CA = Cortical Area; AmygCON = Amygdala Cingulo-Opercular Network connectivity; L = Left; R = Right. For specific r point estimates, see Supplementary Table S1. *p < .05, **p < .01, ***p < .001. (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)

Fig. 3

Reported standardized β estimates for Direct and Indirect effects from original study by Thijssen et al. (2020) and Replication study. A: Original Study with associated 95% CI = ○; Replication Study = ✕. B: Original Study = ○; Replication Study with associated 95% CI = ✕. ACC = Anterior Cingulate Cortex; CA = Cortical Area; CT = Cortical Thickness; Vol = Volume; L/R AmygCON = Left/Right Amygdala Cingulo-Opercular Network connectivity.

Fig. 4

Results of the multiverse analysis expressed as specification curves for all of the 135 models. The blue, gray and red colors indicate whether that standardized β estimate was a significant positive (p < .05), non-significant (p > .05) or a significant negative estimate (p < .05), respectively. Age, sex and race covariates constant across all models. A. Indirect Effect estimates from mediation models; ordered by size and direction for each estimate for an associated X (predictor), Y (outcome) and M (Mediator). B. The associated variables, X, Y and M (Mediator), for each associated effect in the mediation model. PMON = Parental Monitoring; Fact = Factor; Par = Parent; Accept = Child Report of Parent Behavior Inventory (i.e., Acceptance); FES = Family Environment Scale (i.e. conflict; reverse coded); Yth = Youth; FamEnv = Family Environment; Demo = Demographic; Avg IncEdu = Average Parent Reported Income & Education; L/R AmygCON = Left/Right Amygdala Cingulo-Opercular Network connectivity; ACC = Anterior Cingulate Cortex; CT = Cortical Thickness; CA = Cortical Area; PDS = Pubertal Development Scale. (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)

Fig. 5

Reported standardized β estimates for direct and indirect effects from the multiverse analyses for Parental Reported Puberty models only. The Family Environment beta and its associated 95% CI is reported in both Panel A & Panel B to provide reference for how much effects diverge from the original model across different operationalizations of the independent variable. A: estimates for the models using factor derived scores as IV (■ = Family Environment; ▲ = Demographic; ● = Parent; ⬥ = Child). B: estimates for the models using measure derived scores as IV (■ = Family Environment; ○ = FES Youth (i.e., Conflict; reverse coded); + = FES Parent (i.e., Conflict; reverse coded); ✕ = Parental Monitoring; ⬦ = Child Report of Parent Behavior Inventory (i.e., Acceptance); * = Avg Income/Education).ACC = Anterior Cingulate Cortex; CA = Cortical Area; CT = Cortical Thickness; Vol = Volume; L/R AmygCON = Left/Right Amygdala Cingulo-Opercular Network Connectivity. See Supplemental Figs. S8–S9 for all paths.