'Everything is data': towards one big data ecosystem using multiple sources of data on higher education in Indonesia

Abstract

Big data is increasingly being promoted as a game changer for the future of science, as the volume of data has exploded in recent years. Big data characterized, among others, the data comes from multiple sources, multi-format, comply to 5-V's in nature (value, volume, velocity, variety, and veracity). Big data also constitutes structured data, semi-structured data, and unstructured-data. These characteristics of big data formed "big data ecosystem" that have various active nodes involved. Regardless such complex characteristics of big data, the studies show that there exists inherent structure that can be very useful to provide meaningful solutions for various problems. One of the problems is anticipating proper action to students' achievement. It is common practice that lecturer treat his/her class with "one-size-fits-all" policy and strategy. Whilst, the degree of students' understanding, due to several factors, may not the same. Furthermore, it is often too late to take action to rescue the student's achievement in trouble. This study attempted to gather all possible features involved from multiple data sources: national education databases, reports, webpages and so forth. The multiple data sources comprise data on undergraduate students from 13 provinces in Indonesia, including students' academic histories, demographic profiles and socioeconomic backgrounds and institutional information (i.e. level of accreditation, programmes of study, type of university, geographical location). Gathered data is furthermore preprocessed using various techniques to overcome missing value, data categorisation, data consistency, data quality assurance, to produce relatively clean and sound big dataset. Principal component analysis (PCA) is employed in order to reduce dimensions of big dataset and furthermore use K-Means methods to reveal clusters (inherent structure) that may occur in that big dataset. There are 7 clusters suggested by K-Means analysis: 1. very low-risk students, 2. low-risk students, 3. moderate-risk students, 4. fluctuating-risk students, 5. high risk students, 6. very high-risk students and, 7. fail students. Among the clusters unreveal, (1) a gap between public universities and private universities across the three regions in Indonesia, (2) a gap between STEM and non-STEM programmes of study, (3) a gap between rural versus urban, (4) a gap of accreditation status, (5) a gap of quality human resources distribution, etc. Further study, we will use the characteristics of each cluster to predict students' achievement based on students' profiles, and provide solutions and interventions strategies for students to improve their likely success.

Keywords: Big data; Data collection; Data preprocessing; Higher education; Indonesia.

Fig. 1

An example of scree plot of PCA Eigenvalues

Fig. 2

General research framework (modified from [6])

Fig. 3

Infographic representing collection of data on higher education in Indonesia

Fig. 4

Feature correlation matrix

Fig. 5

Principal component index

Fig. 6

Two-dimensional (a) and three-dimensional (b) visualisations of the first two and three principal components

Fig. 7

Elbow plots used to analyse values of k

Fig. 8

Two-dimensional and three-dimensional visualisations with centroids of seven clusters (k-means)

Fig. 9

Boxplots grouped by cluster

Fig. 10

Bar plot analysis by cluster using a seven highest-variance PCs, b 15 highest-variance original features

Fig. 11

Labelling of clusters in dataset