Towards teaching analytics: a contextual model for analysis of students' evaluation of teaching through text mining and machine learning classification

Abstract

Recent trends in educational technology have led to emergence of methods such as teaching analytics (TA) in understanding and management of the teaching-learning processes. Didactically, teaching analytics is one of the promising and emerging methods within the Education domain that have proved to be useful, towards scholastic ways to make use of substantial pieces of evidence drawn from educational data to improve the teaching-learning processes and quality of performance. For this purpose, this study proposed an educational process and data mining plus machine learning (EPDM + ML) model applied to contextually analyze the teachers' performances and recommendations based on data derived from students' evaluation of teaching (SET). The EPDM + ML model was designed and implemented based on amalgamation of the Text mining and Machine learning technologies that builds on the descriptive decision theory, which studies the rationality behind decisions the learners are disposed to make based on the textual data quantification and statistical analysis. To this effect, the study determines pedagogical factors that influences the students' recommendations for their teachers, what role the sentiment and emotions expressed by the students in the SET play in the way they evaluate the teachers by taking into account the gender of the teachers. This includes how to automatically predict what a student's recommendation for the teachers may be based on information about the students' gender, average sentiment, and emotional valence they have shown in the SET. Practically, we applied the Text mining technique to extract the different sentiments and emotions (intensities of the comments) expressed by the students in the SET, and then utilized the quantified data (average sentiment and emotional valence) to conduct an analysis of covariance and Kruskal Wallis Test to determine the influential factors, as well as, how the students' recommendation for the teachers differ by considering the gender constructs, respectively. While a large proportion of the comments that we analyzed (n = 85,378) was classified to be neutral and predominantly interpreted to be positive in nature considering the sentiments (76.4%), and emotional valence (88.2%) expressed by the students. The results of our analysis shows that for the students' comments which contain some kind of positive or negative sentiment (23.6%) and emotional valence (11.8%); that females students recommended the teachers taking into account the sentiments (p = .000). While the males appear to be slightly borderline in terms of emotions (p = .056) and sentiment (p = .077). Also, the EPDM + ML model showed to be a good predictor and efficient method in determining what the students' recommendation scores for the teachers would be, going by the high and acceptable values of the precision (1.00), recall (1.00), specificity (1.00), accuracy (1.00), F1-score (1.00) and zero error-rate (0.00) which we validated using the k-fold cross-validation method, with 63.6% of optimal k-values observed. In theory, we note that not only does the proposed method (EPDM + ML) proves to be useful towards effective analysis of SET and its implications within the educational domain. But can be utilized to determine prominent factors that influences the students' evaluation and recommendation of the teachers, as well as helps provide solutions to the ever-increasingly need to advance and support the teaching-learning processes and/or students' learning experiences in a rapidly changing educational environment or ecosystem.

Keywords: Educational innovation; Higher education; Machine learning; Performance assessment; Teaching analytics; Text mining.

Fig. 1

Educational process and data mining plus machine learning (EPDM + ML) model

Fig. 2

Comments (counts) vs. Ave_Sentiment score broken down by students gender

Fig. 3

Emotional valence scores for the SET comments broken down by students gender

Fig. 4

Overall Emotions expressed by the students for the teachers broken down by gender of the students

Fig. 5

Emotions expressed by the male students for the male vs female teachers

Fig. 6

Emotions expressed by the female students for the male vs female teachers