Temporal Changes in Effect Sizes of Studies Comparing Individuals With and Without Autism: A Meta-analysis

Abstract

Importance: The definition and nature of autism have been highly debated, as exemplified by several revisions of the DSM (DSM-III, DSM-IIIR, DSM-IV, and DSM-5) criteria. There has recently been a move from a categorical view toward a spectrum-based view. These changes have been accompanied by a steady increase in the prevalence of the condition. Changes in the definition of autism that may increase heterogeneity could affect the results of autism research; specifically, a broadening of the population with autism could result in decreasing effect sizes of group comparison studies.

Objective: To examine the correlation between publication year and effect size of autism-control group comparisons across several domains of published autism neurocognitive research.

Data sources: This meta-analysis investigated 11 meta-analyses obtained through a systematic search of PubMed for meta-analyses published from January 1, 1966, through January 27, 2019, using the search string autism AND (meta-analysis OR meta-analytic). The last search was conducted on January 27, 2019.

Study selection: Meta-analyses were included if they tested the significance of group differences between individuals with autism and control individuals on a neurocognitive construct. Meta-analyses were only included if the tested group difference was significant and included data with a span of at least 15 years.

Data extraction and synthesis: Data were extracted and analyzed according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guideline using fixed-effects models.

Main outcomes and measures: Estimated slope of the correlation between publication year and effect size, controlling for differences in methods, sample size, and study quality.

Results: The 11 meta-analyses included data from a total of 27 723 individuals. Demographic data such as sex and age were not available for the entire data set. Seven different psychological and neurologic constructs were analyzed based on data from these meta-analyses. Downward temporal trends for effect size were found for all constructs (slopes: -0.067 to -0.003), with the trend being significant in 5 of 7 cases: emotion recognition (slope: -0.028 [95% CI, -0.048 to -0.007]), theory of mind (-0.045 [95% CI, -0.066 to -0.024]), planning (-0.067 [95% CI, -0.125 to -0.009]), P3b amplitude (-0.048 [95% CI, -0.093 to -0.004]), and brain size (-0.047 [95% CI, -0.077 to -0.016]). In contrast, 3 analogous constructs in schizophrenia, a condition that is also heterogeneous but with no reported increase in prevalence, did not show a similar trend.

Conclusions and relevance: The findings suggest that differences between individuals with autism and those without the diagnosis have decreased over time and that possible changes in the definition of autism from a narrowly defined and homogenous population toward an inclusive and heterogeneous population may reduce our capacity to build mechanistic models of the condition.

Figure 1.. PRISMA Flowchart

Figure 2.. Overview of the Development of Effect Sizes Over Time Within Each of the Constructs

Each point represents an effect size originating from an empirical study. Colors indicate which task or method was used within the study. The black line indicates the fitted linear model. In the analysis of planning and inhibition, method type was defined as the combination of task and outcome metric. Points are colored by task alone for the purpose of visualization. DKEFS indicates Delis-Kaplan Executive Function System; FB-seq, False Belief sequencing task; MASC, Movie for Assessment of Social Cognition; MRI, magnetic resonance imaging; NEPSY, Developmental Neuropsychological Assessment; RMET, Reading the Mind in the Eyes Test; RMVT, Reading the Mind in the Voice Test; and WCST, Wisconsin Card Sorting Task.

Figure 3.. Forest Plot of the Estimated Change in Effect Size per Year