Access to online learning: Machine learning analysis from a social justice perspective

Abstract

Access to education is the first step to benefiting from it. Although cumulative online learning experience is linked academic learning gains, between-country inequalities mean that large populations are prevented from accumulating such experience. Low-and-middle-income countries are affected by disadvantages in infrastructure such as internet access and uncontextualised learning content, and parents who are less available and less well-resourced than in high-income countries. COVID-19 has exacerbated the global inequalities, with girls affected more than boys in these regions. Therefore, the present research mined online learning data to identify features that are important for access to online learning. Data mining of 54,842,787 initial (random subsample n = 5000) data points from one online learning platform was conducted by partnering theory with data in model development. Following examination of a theory-led machine learning model, a data-led approach was taken to reach a final model. The final model was used to derive Shapley values for feature importance. As expected, country differences, gender, and COVID-19 were important features in access to online learning. The data-led model development resulted in additional insights not examined in the initial, theory-led model: namely, the importance of Math ability, year of birth, session difficulty level, month of birth, and time taken to complete a session.

Keywords: COVID-19; Country inequalities; Educational access; Machine learning; Online learning.

Fig. 1

Screenshots from Year 6 Maths Games demo. Screenshots progress from left to right, first the top row, then the bottom row. (See https://www.whizz.com/maths-games/year-6-maths-games.)

Fig. 2

Correlations between potential features and learning outcome (play_count). Transformed data are represented here

Fig. 3

Collective force plot showing the overall effect of all features included in the final model, using absolute mean Shapley values. As the graph progresses to the right, effects of the most important features for each individual learner are shown. Features that push the prediction higher (to the right) are shown in red, and those pushing the prediction lower are in blue. The x-axis shows participant number, ordered by similarity for this plot. Panel A gives a snapshot of the features that generally reduce play_count; Panel B shows a snapshot of features that increase play_count. Transformed data are represented here

Fig. 4

Decision plot of feature importance for global interpretation, using mean absolute Shapley values. The model output value is the learning outcome (play_count). Features that push the prediction higher (to the right) are shown in red, and those pushing the prediction lower are in blue. The fainter a line, the fewer learners it represents. Transformed data are represented here

Fig. 5

Summary plot of feature importance in final model, using mean absolute Shapley values. Features that push the prediction higher (to the right) are shown in red, and those pushing the prediction lower are in blue. Transformed data are represented here

Fig. 6

Bar plot of feature importance of features in the final model, using mean absolute Shapley values. Panel A shows the features ordered from the most important to the least, in the final model. Panel B shows the features are generally ordered in the same way, but with clustering where features are related to each other. Transformed data are represented here

Fig. 7

Line graphs showing how ‘country’ (Kenya, Thailand, and the UK) as well as LMIC status related to ‘access to online learning’ (play_count). Transformed data are represented here

Fig. 8

The role of gender (Male, dummy variable) in predicting access to online learning (play_count)

Fig. 9

Access to online learning (play_count) as the years (markedYear) progress, with the final time point representing the year 2020 (i.e., from the onset of Covid). Transformed data are represented here

Fig. 10

Top five most important features in predicting online learning access (play_count). Transformed data are represented here

Fig. 11

Scatter plot showing how birthYear was related to online learning access (play_count). Untransformed data are represented here

Fig. 12

SHAP interaction values for predicting access to online learning (play_count), from the strongest interaction to the weakest. Only the top 20 interactions are shown here. Transformed data are represented here

Fig. 13

Dependence plots for the six strongest interactants to emerge from the final model

Fig. 14

Dependence plots of the importance of the features that interact with Country (either UK or Kenya) in predicting learning outcomes (lesson mark), according to mean absolute Shapley values. Transformed data are represented here

Fig. 15

The interaction between gender (Male) and timeTaken to complete each lesson. Panel A represents transformed data and relates to feature importance via absolute Shapley values; Panel B represents untransformed data and reflects associations between the variables

Fig. 16

Dependence plots of the importance of the features that interact with Covid, as measured by markedYear, in predicting access to online learning (play_count), according to mean absolute Shapley values. Transformed data are represented here

Fig. 17

The interaction between learner age (pupil_ageQuart) and Covid (i.e., markedYear; pre-covid = 2015–2019, since covid = 2020 onwards) in predicting access to online learning. Untransformed data are represented here

Fig. 18

The interaction between birthMonth and Covid (i.e., markedYear; pre-covid = 2015–2019, since covid = 2020 onwards) in predicting access to online learning (play_count). Untransformed data are represented here

Fig. 19

The interaction between markedWeek and Covid (i.e., markedYear; pre-covid = 2015–2019, since covid = 2020 onwards) in predicting access to online learning (play_count). Untransformed data are represented here

Fig. 20

Dependence plots for the six strongest interactants to emerge from the final model when predicting access to online learning (play_count)

Fig. 21

Coefficients emerging from the regularised regression model with Elastic Net penalty. Transformed data are represented here