Examining Validity for the Pharmacy Affective Domain Situational Judgment Test

Abstract

Objective. With the integration of the affective domain in the pharmacy accreditation standards, it is important for programs to have methods for formatively assessing student competency in these areas. The objective of this study was to examine the validity evidence for a recently developed situational judgment test to assess the affective domain contained in the Accreditation Council for Pharmacy Education (ACPE) Standards 2016.Methods. After pilot testing items in multiple pharmacy schools, a revised 15-item situational judgment test instrument was developed measuring the affective domain as it relates to the pharmacy profession. The resulting Pharmacy Affective Domain Situational Judgment Test (PAD-S) was administered to students at three institutions and internal structure validity evidence was examined using item descriptives, Cronbach alpha, and results from a Rasch measurement model.Results. There were 559 valid responses included in the study. The items were of varying difficulty, with Rasch results confirming the hierarchy of items and suggesting that items were contributing to the measure. The Cronbach alpha was 0.70, suggesting acceptable reliability. However, the reliability items from the Rasch model were lower (0.65 and 0.66), indicating that the ranking of ability was limited, which may be due to fewer items of high difficulty.Conclusion. Overall, the PAD-S performed well as a measure of the affective domain. The PAD-S may be a useful formative instrument as part of a comprehensive assessment plan and may be less resource intensive than other measures.

Keywords: Rasch analysis; affective domain; instrument validity; situational judgment test.

Figure 1.

Outline of the process used to develop and evaluate the validity of a Pharmacy Affective Domain Situational Judgment Test (PAD-S).