Critical Analysis of Primary Literature in a Master's-Level Class: Effects on Self-Efficacy and Science-Process Skills

Abstract

The ability to think analytically and creatively is crucial for success in the modern workforce, particularly for graduate students, who often aim to become physicians or researchers. Analysis of the primary literature provides an excellent opportunity to practice these skills. We describe a course that includes a structured analysis of four research papers from diverse fields of biology and group exercises in proposing experiments that would follow up on these papers. To facilitate a critical approach to primary literature, we included a paper with questionable data interpretation and two papers investigating the same biological question yet reaching opposite conclusions. We report a significant increase in students' self-efficacy in analyzing data from research papers, evaluating authors' conclusions, and designing experiments. Using our science-process skills test, we observe a statistically significant increase in students' ability to propose an experiment that matches the goal of investigation. We also detect gains in interpretation of controls and quantitative analysis of data. No statistically significant changes were observed in questions that tested the skills of interpretation, inference, and evaluation.

Figure 1.

Students’ demographics and career aspirations, based on an anonymous precourse survey. n = 28 students. (A) Quarter in the master’s program. (B) Students’ ethnic background. (C) UCSD biology major affiliation as undergraduate. (D) Students’ current career aspirations. Students could select all career options they are currently considering from a list of options, so the sum of responses exceeds the total number of students.

Figure 2.

Course modules and individual module structure. (A) Papers discussed in the three course modules. The first two modules focused on one paper, while the third module focused on two papers that addressed the same experimental question but reached different conclusions. (B) The structure of a module. Each module consisted of five class meetings. Assignments outside class included: written analysis of three key experiments (submitted individually) and a follow-up experimental proposal that was submitted by groups of three to five students.

Figure 3.

Students’ self-efficacy in science-process skills in the context of primary literature. Twenty-eight pairs of matched responses from anonymous surveys given at the beginning (Pre) and end (Post) of the quarter were analyzed. A list of survey questions that were grouped into the categories of interpretation and inference, evaluation, and experimental design is provided in Table 1. “Percent responses” refers to the frequency of a specific rating (poor, adequate, etc.) among all responses to the questions that were grouped into the same category (interpretation and inference, evaluation, or experimental design).

Figure 4.

Analysis of science-process tests. Thirty-three pairs of matched tests from the beginning (Pre) and end (Post) of the quarter were evaluated. (A) Student performance in the categories of interpretation, inference, evaluation, and experimental design, before and after instruction. Small but statistically significant gains were observed in the experimental design category (p = 0.039, Cohen’s d = 0.379). (B) Average pre- and posttest score in the category of appropriateness. The increase in the posttest scores was statistically significant (p = 0.005, Cohen’s d = 0.526). (C) Average pre- and posttest score in the category of interpretation of controls. The increase in the posttest scores was statistically significant (p = 0.049, Cohen’s d = 0.37).

Figure 5.

Average pre- and posttest scores in the quantitative data analysis category. The increase in the posttest scores was statistically significant (p = 0.002, Cohen’s d = 0.728, n = 33 pairs of pre- and posttests).