Entering Research Learning Assessment (ERLA): Validity Evidence for an Instrument to Measure Undergraduate and Graduate Research Trainee Development

Abstract

Expanding the scope of previous undergraduate research assessment tools, the Entering Research Learning Assessment (ERLA) measures undergraduate and graduate research trainee learning gains in the seven areas of trainee development in the evidence-based Entering Research conceptual framework: Research Comprehension and Communication Skills, Practical Research Skills, Research Ethics, Researcher Identity, Researcher Confidence and Independence, Equity and Inclusion Awareness and Skills, and Professional and Career Development Skills. In this paper, we present multiple sources of validity evidence for the ERLA trainee self-assessment and mentor assessment of trainee learning gains. Evidence of internal structure of the initial scales via exploratory factor analysis (N_trainees = 193; N_mentors = 130) revealed seven factors that align with the Entering Research conceptual framework. Validity evidence for internal structure using confirmatory factor analysis, convergent validity, and evidence of internal consistency for the revised scale were examined with a larger sample (N_trainees = 489; N_mentors = 256). Evidence of internal structure and alignment for a paired version of the ERLA was also examined with a subset of the original sample (N = 121 pairs). Each analysis revealed acceptable model-data fit. Guidance on using the ERLA instruments and interpreting their scores is presented.

FIGURE 1.

Development of Entering Research Learning Assessment (ERLA).

FIGURE 2.

Degree of alignment between trainees’ self-reported gains and mentors’ assessment of trainee gains (n = 121). Alignment was calculated by subtracting the trainee’s score for each subscale from the mentor’s score for each subscale. RCC, Research Comprehension and Communication Skills; PRS, Practical Research Skills; RE, Research Ethics; RID, Researcher Identity; RCI, Researcher Confidence and Independence; EIA, Equity and Inclusion Awareness and Skills; PDS, Professional and Career Development Skills.