School-based interventions for reducing disciplinary school exclusion: a systematic review

Abstract

This Campbell systematic review examines the impact of interventions to reduce exclusion from school. School exclusion, also known as suspension in some countries, is a disciplinary sanction imposed by a responsible school authority, in reaction to students' misbehaviour. Exclusion entails the removal of pupils from regular teaching for a period during which they are not allowed to be present in the classroom (in-school) or on school premises (out-of-school). In some extreme cases the student is not allowed to come back to the same school (expulsion). The review summarises findings from 37 reports covering nine different types of intervention. Most studies were from the USA, and the remainder from the UK. Included studies evaluated school-based interventions or school-supported interventions to reduce the rates of exclusion. Interventions were implemented in mainstream schools and targeted school-aged children from four to 18, irrespective of nationality or social background. Only randomised controlled trials are included. The evidence base covers 37 studies. Thirty-three studies were from the USA, three from the UK, and for one study the country was not clear. School-based interventions cause a small and significant drop in exclusion rates during the first six months after intervention (on average), but this effect is not sustained. Interventions seemed to be more effective at reducing some types of exclusion such as expulsion and in-school exclusion. Four intervention types - enhancement of academic skills, counselling, mentoring/monitoring, and skills training for teachers - had significant desirable effects on exclusion. However, the number of studies in each case is low, so this result needs to be treated with caution. There is no impact of the interventions on antisocial behaviour. Variations in effect sizes are not explained by participants' characteristics, the theoretical basis of the interventions, or the quality of the intervention. Independent evaluator teams reported lower effect sizes than research teams who were also involved in the design and/or delivery of the intervention.

Plain language summary: Interventions can reduce school exclusion but the effect is temporary: Some interventions - enhancement of academic skills, counselling, mentoring/monitoring, and skills training for teachers - appear to have significant effects on exclusion.The review in brief: Interventions to reduce school exclusion are intended to mitigate the adverse effects of this school sanction. Some approaches, namely those involving enhancement of academic skills, counselling, mentoring/monitoring and those targeting skills training for teachers, have a temporary effect in reducing exclusion. More evaluations are needed to identify the most effective types of intervention; and whether similar effects are also found in different countries.What is this review about?: School exclusion is associated with undesirable effects on developmental outcomes. It increases the likelihood of poor academic performance, antisocial behavior, and poor employment prospects. This school sanction disproportionally affects males, ethnic minorities, those who come from disadvantaged economic backgrounds, and those with special educational needs.This review assesses the effectiveness of programmes to reduce the prevalence of exclusion.What are the main findings of this review?: What studies are included? Included studies evaluated school-based interventions or school-supported interventions to reduce the rates of exclusion. Interventions were implemented in mainstream schools and targeted school-aged children from four to 18, irrespective of nationality or social background. Only randomised controlled trials are included.The evidence base covers 37 studies. Thirty-three studies were from the USA, three from the UK, and for one study the country was not clear.School-based interventions cause a small and significant drop in exclusion rates during the first six months after intervention (on average), but this effect is not sustained. Interventions seemed to be more effective at reducing some types of exclusion such as expulsion and in-school exclusion.Four intervention types - enhancement of academic skills, counselling, mentoring/ monitoring, and skills training for teachers - had significant desirable effects on exclusion. However, the number of studies in each case is low, so this result needs to be treated with caution.There is no impact of the interventions on antisocial behaviour.Variations in effect sizes are not explained by participants' characteristics, the theoretical basis of the interventions, or the quality of the intervention. Independent evaluator teams reported lower effect sizes than research teams who were also involved in the design and/or delivery of the intervention.What do the findings of this review mean?: School-based interventions are effective at reducing school exclusion immediately after, and for a few months after, the intervention (6 months on average). Four interventions presented promising and significant results in reducing exclusion, that is, enhancement of academic skills, counselling, mentoring/monitoring, skills training for teachers. However, since the number of studies for each sub-type of intervention was low, we suggest these results should be treated with caution.Most of the studies come from the USA. Evaluations are needed from other countries in which exclusion is common. Further research should take advantage of the possibility of conducting cluster-randomised controlled trials, whilst ensuring that the sample size is sufficiently large.How up-to-date is this review?: The review authors searched for studies published up to December 2015. This Campbell systematic review was published in January 2018.

Executive summary/abstract: BACKGROUND: Schools are important institutions of formal social control (Maimon, Antonaccio, & French, 2012). They are, apart from families, the primary social system in which individuals are socialised to follow specific codes of conduct. Violating these codes of conduct may result in some form of punishment. School punishment is normally accepted by families and students as a consequence of transgression, and in that sense school isoften the place where children are first introduced to discipline, justice, or injustice (Whitford & Levine-Donnerstein, 2014).A wide range of punishments may be used in schools, from verbal reprimands to more serious actions such as detention, fixed term exclusion or even permanent exclusion from the mainstream education system. It must be said that in some way, these school sanctions resemble the penal system and its array of alternatives to punish those that break the law.School exclusion, also known as suspension in some countries, is defined as a disciplinary sanction imposed by a responsible school authority, in reaction to students' misbehaviour. Exclusion entails the removal of pupils from regular teaching for a period during which they are not allowed to be present in the classroom or, in more serious cases, on school premises.Based on the previous definition, this review uses school exclusion and school suspension as synonyms, unless the contrary is explicitly stated. Most of the available research has found that exclusion correlates with subsequent negative sequels on developmental outcomes. Exclusion or suspension of students is associated with failure within the academic curriculum, aggravated antisocial behaviour, and an increased likelihood of involvement with punitive social control institutions (i.e., the Juvenile Justice System). In the long-term, opportunities for training and employment seem to be considerably reduced for those who have repeatedly been excluded. In addition to these negative correlated outcomes, previous evidence suggest that the exclusion of students involves a high economic cost for taxpayers and society.Research from the last 20 years has concluded quite consistently that this disciplinary measure disproportionally targets males, ethnic minorities, those who come from disadvantaged economic backgrounds, and those presenting special educational needs. In other words, suspension affects the most vulnerable children in schools.Different programmes have attempted to reduce the prevalence of exclusion. Although some of them have shown promising results, so far, no comprehensive systematic review has examined these programmes' overall effectiveness.OBJECTIVES: The main goal of the present research is to systematically examine the available evidence for the effectiveness of different types of school-based interventions aimed at reducing disciplinary school exclusion. Secondary goals include comparing different approaches and identifying those that could potentially demonstrate larger and more significant effects.The research questions underlying this project are as follows: Do school-based programmes reduce the use of exclusionary sanctions in schools?Are some school-based approaches more effective than others in reducing exclusionary sanctions?Do participants' characteristics (e.g., age, gender, ethnicity) affect the impact of school-based programmes on exclusionary sanctions in schools?Do characteristics of the interventions, implementation, and methodology affect the impact of school-based programmes on exclusionary sanctions in schools? SEARCH METHODS: The authors conducted a comprehensive search to locate relevant studies reporting on the impact of school-based interventions on exclusion from 1980 onwards. Twenty-seven different databases were consulted, including databases that contained both published and unpublished literature. In addition, we contacted researchers in the field of school-exclusion for further recommendations of relevant studies; we also assessed citation lists from previous systematic and narrative reviews and research reports. Searches were conducted from September 1 to December 1, 2015.SELECTION CRITERIA: The inclusion and exclusion criteria for manuscripts were defined before we started our searches. To be eligible, studies needed to have: evaluated school-based interventions or school-supported interventions intended to reduce the rates of suspension; seen the interventions as an alternative to exclusion; targeted school-aged children from four to 18 in mainstream schools irrespective of nationality or social background; and reported results of interventions delivered from 1980 onwards. In terms of methodological design, we included randomised controlled trialsonly, with at least one experimental group and onecontrol or placebo group.DATA COLLECTION AND ANALYSIS: Initial searches produced a total of 42,749 references from 27 different electronic databases. After screening the title, abstract and key words, we kept 1,474 relevant hits. 22 additional manuscripts were identified through other sources (e.g., assessment of citation lists, contribution of authors). After removing duplicates, we ended up with a total of 517 manuscripts. Two independent coders evaluated each report, to determine inclusion or exclusion.The second round of evaluation excluded 472 papers, with eight papers awaiting classification, and 37 studies kept for inclusion in meta-analysis. Two independent evaluators assessed all the included manuscripts for risk of quality bias by using EPOC tool.Due to the broad scope of our targeted programmes, meta-analysis was conducted under a random-effect model. We report the impact of the intervention using standardised differences of means, 95% confidence intervals along with the respective forest plots. Sub-group analysis and meta-regression were used for examining the impact of the programme. Funnel plots and Duval and Tweedie's trim-and-fill analysis were used to explore the effect of publication bias.RESULTS: Based on our findings, interventions settled in school can produce a small and significant drop in exclusion rates (SMD=.30; 95% CI .20 to .41; p<.001). This means that those participating in interventions are less likely to be suspended than those allocated to control/placebo groups. These results are based on measures of impact collected immediately during the first six months after treatment (on average). When the impact was tested in the long-term (i.e., 12 or more months after treatment), the effects of the interventions were not sustained. In fact, there was a substantive reduction in the impact of school-based programmes (SMD=.15; 95%CI -.06 to .35), and it was no longer statistically significant.We ran analysis testing the impact of school-based interventions on different types of exclusion. Evidence suggests that interventions are more effective at reducing expulsion and in-school exclusion than out-of-school exclusion. In fact, the impact of intervention in out-of-school exclusion was close to zero and not statistically significant.Nine different types of school-based interventions were identified across the 37 studies included in the review. Four of them presented favourable and significant results in reducing exclusion (i.e., enhancement of academic skills, counselling, mentoring/monitoring, skills training for teachers). Since the number of studies for each sub-type of intervention was low, we suggest that results should be treated with caution.A priori defined moderators (i.e., participants' characteristics, the theoretical basis of the interventions, and quality of the intervention)showed not to be effective at explaining the heterogeneity present in our results. Among three post-hoc moderators, the role of the evaluator was found to be significant: independent evaluator teams reported lower effect sizes than research teams who were also involved in the design and/or delivery of the intervention.Two researchers independently evaluated the quality of the evidence involved in this review by using the EPOC tool. Most of the studies did not present enough information for the judgement of quality bias.AUTHORS' CONCLUSIONS: The evidence suggests that school-based interventions are effective at reducing school exclusion immediately after, and for a few months after, the intervention. Some specific types of interventions show more promising and stable results than others, namely those involving mentoring/monitoring and those targeting skills training for teachers. However, based on the number of studies involved in our calculations, we suggest that results must be cautiously interpreted. Implications for policy and practice arising from our results are discussed.

Figure 1

PRISMA flow chart of searches

Figure 2

Risk of bias in included studies based on EPOC risk of bias tool Note. Each of the eight evaluated criteria have been assigned one of three possible alternatives: high risk= HR; low risk=LR and unclear risk= UR, as expressed in the first column underneath the graph.

Figure 3

Forest plot of the effect sizes for the impact of school‐based programmes on school exclusion

Figure 4

Forest plot of the effect sizes for the impact of school‐based programmes on school exclusion: long‐term effects

Figure 5

Forest plot of the effect sizes for the impact of school‐based programmes on school exclusion: post‐treatment (seven studies only)

Figure 6

Forest plot of the effect sizes for the impact of school‐based programmes on school exclusion: follow‐up (seven studies only)

Figure 7

Forest plot of the effect sizes for the impact of school‐based programmes on in‐school exclusion

Figure 8

Forest plot of the effect sizes for the impact of school‐based programmes on out‐of‐school

Figure 9

Forest plot of the effect sizes for the impact of school‐based programmes on school expulsion

Figure 10

Forest plot of the effect sizes for the impact of school‐based programmes on general school suspension

Figure 11

Forest plot of the effect sizes for the impact of school‐based programmes on antisocial behaviour

Figure 12

R‐squared graphic. Proportion of the variance explained by the role of the evaluator

Figure 13

Funnel plot of standard error by standard differences in means