Crossing Boundaries: Steps Toward Measuring Undergraduates' Interdisciplinary Science Understanding

Abstract

A desired outcome of education reform efforts is for undergraduates to effectively integrate knowledge across disciplines in order to evaluate and address real-world issues. Yet there are few assessments designed to measure if and how students think interdisciplinarily. Here, a sample of science faculty were surveyed to understand how they currently assess students' interdisciplinary science understanding. Results indicate that individual writing-intensive activities are the most frequently used assessment type (69%). To understand how writing assignments can accurately assess students' ability to think interdisciplinarily, we used a preexisting rubric, designed to measure social science students' interdisciplinary understanding, to assess writing assignments from 71 undergraduate science students. Semistructured interviews were conducted with 25 of those students to explore similarities and differences between assignment scores and verbal understanding of interdisciplinary science. Results suggest that certain constructs of the instrument did not fully capture this competency for our population, but instead, an interdisciplinary framework may be a better model to guide assessment development of interdisciplinary science. These data suggest that a new instrument designed through the lens of this model could more accurately characterize interdisciplinary science understanding for undergraduate students.

FIGURE 1.

Box plots compare student overall mean construct scores (n = 71). Nonidentical letters above bars represent significant (p < 0.05) differences among construct scores (as determined by ANOVA and post hoc pairwise comparisons using Tukey’s HSD). A one-way Welch’s ANOVA detected a significant difference between mean construct scores (F(3, 280) = 6.149, p = 0.00057, η² = 0.062). Tukey’s post hoc analyses reveal that students scored significantly higher on purposefulness than integration and critical awareness (p = 0.0025 and p = 0.0139, respectively), with no significant differences between the latter two constructs. Students performed significantly better on disciplinary grounding than integration (p = 0.0185), with no significant differences between disciplinary grounding and purposefulness. Box: 25th to 75th percentile; bars: minimum and maximum values. The error bars represent the standard error of the mean.

FIGURE 2.

Box plots compare students’ mean essay scores across four upper-division courses (n = 71). Nonidentical letters above bars represent significant (p < 0.05) differences among courses (as determined by ANOVA and post hoc pairwise comparisons using Tukey’s HSD). One-way ANOVA revealed a significant difference between mean construct scores (F(3, 67) = 3.691, p = 0.016, η² = 0.142). A Tukey’s post hoc test indicated a significant difference in mean essay scores between Chemical Ecology and Environmental Ecology (p = 0.0187), with no significant differences between other courses. Box: 25th to 75th percentile; bars: minimum and maximum values.

FIGURE 3.

Comparison of mean construct scores for students enrolled in four courses (n = 71). Nonidentical letters above bars represent significant (p < 0.05) differences among courses within each construct (as determined by ANOVA and post hoc pairwise comparisons using Tukey’s HSD). One-way ANOVA indicated a significant difference between course scores based on the constructs disciplinary grounding (F(3, 68) = 14.5, p < 0.0001, η² = 0.329), integration (F(3, 68) = 19.2, p < 0.0001, η² = 0.401), and critical awareness (F(3, 68) = 8.38, p = 0.0003, η² = 0.187; Welch’s ANOVA for unequal variances reported based on significant Levene’s test for integration and critical awareness). Tukey’s post hoc tests: (A) construct purposefulness: no significant differences in student scores across courses; (B) construct disciplinary grounding: students in Chemical Ecology, Biochemical Virology, and Plant Systematics score significantly higher than students in Environmental Restoration (p < 0.0001, p = 0.0024, and p = 0.0435, respectively); (C) construct integration: students enrolled in Biochemical Virology and Chemical Ecology significantly outperformed students in Plant Systematics (p = 0.0207 and p = 0.0138, respectively) and in Environmental Restoration (p < 0.0001 for both courses); (D) construct critical awareness: students in Chemical Ecology and Environmental Restoration scored significantly higher than students in Plant Systematics (p = 0.006 and p = 0.016, respectively). The error bars represent the standard error of the mean.

FIGURE 4.

Numeric construct scores, (1) naïve, (2) novice, (3) apprentice, and (4) mastery, matched with same-student binary interview score (yes, no). (A) Disciplinary grounding, (B) integration, and (C) critical awareness. Bubble size corresponds to the number of students who obtained a given construct and interview score (i.e., larger bubbles indicate a greater number of students who received a particular matched score).