Predictive validity of the MCAT as a function of undergraduate institution

Abstract

The question of whether the predictive ability of the Medical College Admission Test (MCAT) differed for students from different undergraduate institutions was addressed by the authors in the study reported here. Two groups of students were studied: group 1 comprised 1,859 students who entered Jefferson Medical College of Thomas Jefferson University between 1964 and 1977, and group 2 consisted of 999 students who entered the college between 1978 and 1982. Ten undergraduate institutions with at least 20 matriculants in each group were selected for analysis. Group 1 students had taken the old version and group 2 the new version of the MCAT. Scores on the Science subtest of the old MCAT were used as the predictor for group 1, and scores on the Science Problems subtest of the new MCAT were used as the predictor for group 2. First-year and second-year medical school grade-point averages and total scores on the Part I and Part II examinations of the National Board of Medical Examiners were the performance measures used. Validity coefficients were derived of the predictive value of the MCAT scores at each of the 10 undergraduate institutions. Striking differences were found in validity coefficients among these institutions. These differences raise questions about the predictive validity of the MCAT when scores for different undergraduate institutions are combined in deriving the coefficients. Possible explanations, implications for admissions decisions and validity studies, and limitations of these findings are discussed.