Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 May 12;6(5):e03917.
doi: 10.1016/j.heliyon.2020.e03917. eCollection 2020 May.

A User Interface (UI) and User eXperience (UX) evaluation framework for cyberlearning environments in computer science and software engineering education

Hakam W Alomari  1 , Vijayalakshmi Ramasamy  1 , James D Kiper  1 , Geoff Potvin  2
Affiliations

A User Interface (UI) and User eXperience (UX) evaluation framework for cyberlearning environments in computer science and software engineering education

Hakam W Alomari et al. Heliyon. .

Abstract

Despite the widespread availability and increasing use of cyberlearning environments, there remains a need for more research about their usefulness in undergraduate education, particularly in STEM education. The process of evaluating the usefulness of a cyberlearning environment is an essential measure of its success and is useful in assisting the design process and ensuring user satisfaction. Unfortunately, there are relatively few empirical studies that provide a comprehensive test of the usefulness of cyberlearning in education. Additionally, there is a lack of standards upon whose usefulness evaluators agree. In this research, we present multiple user studies that can be used to assess the usefulness of a cyberlearning environment used in Computer Science and Software Engineering courses through testing its usability and measuring its utility using user interface and user experience evaluations. Based on these assessments, we propose an evaluation framework to evaluate cyberlearning environments. To help illustrate the framework utility and usability evaluations, we explain them through an example SEP-CyLE (Software Engineering and Programming Cyberlearning Environment). The evaluation techniques used are cognitive walkthroughs with a think-aloud protocol and a heuristic evaluation survey. We further use a network-based analysis to find the statistically significant correlated responses in the heuristic evaluation survey with regard to the students' perceptions of using SEP-CyLE. Our goal is to improve cyberlearning practice and to emphasize the need for evaluating cyberlearning environments with respect to its designated tasks and its users using UI/UX evaluations. Our experiments demonstrated participants were able to utilize SEP-CyLE efficiently to accomplish the tasks we posed to them and to enhance their software development concepts, specifically, software testing. We discovered areas of improvement in the visibility and navigation of SEP-CyLE's current design. We provide our recommendations for improving SEP-CyLE and provide guidance and possible directions for future research on designing cyberlearning environments for computer education.

Keywords: Computer science; Cyberlearning environment; STEM education; Usability evaluation; User experience; User interface.

PubMed Disclaimer

Figures

Figure 1
Figure 1
SEP-CyLE's Professor view and Student view.
Figure 2
Figure 2
Concepts and attributes of our UI/UX evaluation framework of a cyberlearning enviroment - SEP-CyLE is used as a case study.
Figure 3
Figure 3
Pretest and Posttest scores for Control Group.
Figure 4
Figure 4
Pretest and Posttest scores for Treatment Group.
Figure 5
Figure 5
Students' mean scores measuring their overall reactions to SEP-CyLE.
Figure 6
Figure 6
Students' mean scores measuring usefulness of testing tutorials and collaborative learning environment in SEP-CyLE.
Figure 7
Figure 7
Rubric evaluation scores in % grouped by category name (Home, LOs, Quizzes, and Tutorials), and by task number (T1 - T12) being evaluated.
Figure 8
Figure 8
Histogram of difficulty levels of various categories of tasks.
Figure 9
Figure 9
Significant difference between the means of (DL0 & DL1), (DL1 & DL2), and (DL0 & DL2) groups.
Figure 10
Figure 10
Percentages of participants' responses.
Figure 11
Figure 11
Medians from heuristic evaluation by question number (columns) and by question groupings (dashed line).
Figure 12
Figure 12
Statistically Significant Heuristic Survey Graph. 21 survey questions - nodes, pair-wise associations between the responses estimated using the statistical Pearson Correlation (r) - edges (a) Subnetwork represents significant correlations of responses between 12 questions; a 5-clique shown in circle with 5 questions of similar responses; High negative correlation of responses between Q10 and Q18 (b) Sparsely Connected Subnetwork of 4 questions showing varied responses (c) Disconnected subnetwork with very dissimilar responses to 5 questions.
Figure 13
Figure 13
Final Scores comparison between treatment group and control group.
See this image and copyright information in PMC

References

    1. Borgman C.L., Abelson H., Dirks L., Johnson R., Koedinger K.R., Linn M.C., Lynch C.A., Oblinger D.G., Pea R.D., Salen K. 2008. Fostering Learning in the Networked World: the Cyberlearning Opportunity and challenge. A 21st century Agenda for the National Science Foundation.
    1. Roschelle J., Martin W., Ahn J., Schank P.E. 2017. Cyberlearning Community Eport: the State of Cyberlearning and the Future of Learning with Technology.
    1. Clarke P.J., Allen A.A., King T.M., Jones E.L., Natesan P. Proceedings of the ACM International Conference Companion on Object Oriented Programming Systems Languages and Applications Companion, OOPSLA ’10, ACM, New York, NY, USA. 2010. Using a webbased repository to integrate testing tools into programming courses; pp. 193–200.
    1. Clarke P.J., Davis D.L., Chang-Lau R., King T.M. Impact of using tools in an undergraduate software testing course supported by wrestt. ACM Trans. Comput. Educ. 2017;17(4):18:1–18:28.
    1. Clarke P.J., Davis D., King T.M., Pava J., Jones E.L. Integrating testing into software engineering courses supported by a collaborative learning environment. Trans. Comput. Educ. 2014;14(3):18:1–18:33.

LinkOut - more resources