Evaluation of two CD-ROMs from a series on cell biology

Abstract

Two CD-ROMs from a series dealing with various major aspects of cell biology are evaluated in this paper using quantitative and qualitative approaches. The findings delimit similarities and differences of the two CD-ROMs and shed light on how the programs could be used in the learning process and how they should not be. The overall impression, as well as the graphical and technical features, received a predominantly good rating. The defined target groups were reached (e.g., students in secondary schools), different learning approaches were supported (e.g., discovery and autonomous learning), the CD-ROMs' usability was assessed as being easy and intuitive, and the majority of the evaluators were satisfied with the level of interactivity. Navigational problems encountered in CD-ROM 1 were overcome by a successful implementation of new navigational functions in CD-ROM 2. Most students found the CD-ROM to be a suitable complement to, or an extension of, their lessons. We conclude that many, but not all of the requirements for the various stages of the learning process could be satisfied with the existing CD-ROMs. The requirements not met are discussed to obtain insights that could help to improve the production of multimedia learning material. The use of quantitative and qualitative approaches in the evaluation of learning modules is discussed, as the study began by collecting and analyzing anecdotal reviews and was then extended to include a qualitative evaluation.

Figure 1

Virtual laboratory (CD-ROM 2).

Figure 2

3-D animation (CD-ROM 1).

Figure 3

Short videos with microscopic shots (CD-ROM 2).

Figure 4

Navigational differences. Two clicks are necessary to access the organizational chart index on CD-ROM 1: first, click on the curved arrow on top of the test tube (1), then click on “Index” (2). The organizational chart index (“Compass”) in CD-ROM 2 is always visible and accessible with one click (3).

Figure 5

Your opinion of the CD-ROM.

Figure 6

Three navigational approaches on the start page of CD-ROM 2—a compass (sitemap), an explorative tour, and two guided tours.

Figure 7

Ratings of the CD-ROM 2. The rating scale ranges, in five steps, from 2 (positive assessment; “I agree with the statement”) to −2 (negative assessment; “I don't agree with the statement”). The distribution of assessments for each question is given by a Box–Whisker Plot indicating the median (thick vertical line), the interquatile range (gray box), and the range (horizontal line).

Figure 8

Impact of prior biological knowledge and computer literacy on students' response to Statement 4, “I plan to use the CD-ROM in the future” (see Figure 5). Pretest scores of students who disagreed with this statement (−2 = “I disagree”, −1 = “I more or less disagree”) are compared to scores of students who agreed with this statement (1 = “I more or less agree”, 2 = “I agree”). The distribution of posttest scores is given by a Box–Whisker plot indicating the median (thick horizontal line), the interquatile range (gray box), and the range (vertical line).