Teaching and Incentives: Substitutes or Complements?

Abstract

Interventions to promote learning are often categorized into supply- and demand-side approaches. In a randomized experiment to promote learning about COVID-19 among Mozambican adults, we study the interaction between a supply and a demand intervention, respectively: teaching via targeted feedback, and providing financial incentives to learners. In theory, teaching and learner-incentives may be substitutes (crowding out one another) or complements (enhancing one another). Experts surveyed in advance predicted a high degree of substitutability between the two treatments. In contrast, we find substantially more complementarity than experts predicted. Combining teaching and incentive treatments raises COVID-19 knowledge test scores by 0.5 standard deviations, though the standalone teaching treatment is the most cost-effective. The complementarity between teaching and incentives persists in the longer run, over nine months post-treatment.

Keywords: Africa; COVID-19; Cost-effectiveness; D90; Education; I10; I21; Learning; Mozambique; Teaching.

Figure 1:. Study Timeline

Notes: Pre-analysis plan uploaded and treatments randomly assigned immediately prior to start of baseline survey, on Aug. 25, 2020. Treatments implemented immediately following baseline survey on same phone call. There was at least a three week gap between baseline and endline survey for any given study participant. Not depicted is the post-endline survey implemented between June 30 and August 30, 2021 that we use in the long-run analysis described in Section 5.4.

Figure 2:. Distributions of Expert Predictions of Treatment Effects and Complementarity Parameter

Notes: Probability density functions of predicted treatment effects of 67 experts surveyed prior to results being publicized (survey closing date Jan. 2, 2021). Experts predicted effects of “Incentive”, “Teaching”, and “Incentive plus Teaching” (“Joint”) treatments on COVID-19 knowledge test score (fraction of questions answered correctly). Expert-predicted $\lambda$ values are calculated from each expert’s predictions. Mean of expert-predicted $\lambda$ values is $\tilde{\lambda }=-0.0265$. Smoothing uses Epanechnikov kernel with bandwidth 0.9924.

Figure 3:. Treatment Effects and Test of Complementarity Parameter λ Against Benchmark Values

Notes: Panel (a) dependent variable on y-axis is the Teaching-Eligible test score (share of correct answers to knowledge questions asked at baseline and hence eligible for all treatments). Panel (b) dependent variable is Teaching-Ineligible test score (share of correct answers to knowledge questions NOT asked at baseline and hence NOT eligible for the Teaching intervention). Bars in first three columns display regression coefficients representing treatment effects (and 95% confidence intervals) for “Incentive”, “Teaching”, and “Incentive plus Teaching” (“Joint”) treatments. Floating solid horizontal lines in fourth and fifth columns display “Incentive plus Teaching” (“Joint”) treatment effects that would be implied by different benchmark values of complementarity parameter $\lambda$. Difference between values in 3rd and 4th columns is actual estimated complementarity parameter, $\widehat{\lambda }$; the test that this difference is equal to zero tests the null that $\lambda =0$. Difference between values in 3rd and 5th columns is difference between $\widehat{\lambda }$ and mean expert prediction, $\tilde{\lambda }=-0.0265$; the test that this difference is equal to zero tests the null that $\lambda =-0.0265$.

Figure 4:. Cumulative Distribution Functions of Test Score by Treatment Group

Notes: Panel (a) dependent variable on y-axis is the Teaching-Eligible test score (share of correct answers to knowledge questions asked at baseline and hence eligible for all treatments). Panel (b) dependent variable is Teaching-Ineligible test score (share of correct answers to knowledge questions NOT asked at baseline and hence NOT eligible for the Teaching intervention). Figure displays cumulative distribution functions (CDFs) of test scores in “Control”, “Incentive”, “Teaching”, and “Incentive plus Teaching” (“Joint”) treatment groups.