Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Mar 5;15(2):94-104.
doi: 10.21500/20112084.5657. eCollection 2022 Jul-Dec.

Assessing Language Skills Using Diagnostic Classification Models: An Example Using a Language Instrument

Georgios D Sideridi  1   2 , Ioannis Tsaousis  3 , Khaleel Al-Harbi  4
Affiliations

Assessing Language Skills Using Diagnostic Classification Models: An Example Using a Language Instrument

Georgios D Sideridi et al. Int J Psychol Res (Medellin). .

Abstract
in English, Spanish

The primary purpose of the present study is to inform and illustrate, using examples, the use of Diagnostic Classification Models (DCMs) for the assessment of skills and competencies in cognition and academic achievement. A secondary purpose is to compare and contrast traditional and contemporary psychometrics for the measurement of skills and competencies. DCMs are described along the lines of other psychometric models within the Confirmatory Factor Analysis tradition such as the bifactor model and the known mixture models that are utilized to classify individuals into subgroups. The inclusion of interaction terms and constraints along with its confirmatory nature enables DCMs to accurately assess the possession of skills and competencies. The above is illustrated using an empirical dataset from Saudi Arabia (n = 2642), in which language skills are evaluated on how they conform to known levels of competency based on the CEFR (Council of Europe, 2001) using the English Proficiency Test (EPT).

El propósito principal del presente estudio fue informar e ilustrar, mediante ejemplos, el uso de Modelos de Clasificación Diagnóstica (DCM) para la evaluación de habilidades y competencias en cognición y rendimiento académico. Un propósito secundario fue comparar y contrastar la psicometría tradicional y contemporánea para la medición de habilidades y competencias. Los DCM se describen siguiendo las líneas de otros modelos psicométricos dentro de la tradición del Análisis Factorial Confirmatorio, como el modelo bifactor y los conocidos modelos mixtos que se utilizan para clasificar a los individuos en subgrupos. La inclusión de términos y restricciones de interacción junto con su naturaleza confirmatoria permite a los DCM evaluar con precisión la posesión de habilidades y competencias. Lo anterior se ilustra utilizando un conjunto de datos empíricos de Arabia Saudita (n = 2642), que evalúan cómo las habilidades lingüísticas se ajustan a los niveles de competencia conocidos, basados en el MCER (Council of Europe, 2001).

Keywords: CEFR; Diagnostic Classification Models; bifactor models; cognitive Diagnostic Models; language.

PubMed Disclaimer

Conflict of interest statement

Conflict of interests: The authors have declared that there is no conflict of interest.

Figures

Figure 1
Figure 1. The logic of diagnostic classification models
Figure 2
Figure 2. Traditional and contemporary models for the assessment of skills and competencies. Horizontal lines within boxes reflect thresholds of categorical variables
Figure 3
Figure 3. Two-factor correlated model for the assessment of A1 and A2 skills of the EPT instrument (upper panel) and bifactor model extending the 2-factor correlated model with the inclusion of a general factor and two specific ones.
Figure 4
Figure 4. Diagnostic cognitive model for the assessment of pre-A1, A1, and A2 skills and competencies of the EPT measure
Figure 5
Figure 5. Comparisons between person estimates of ability as based on the factor model and the DCM model’s estimates. Values on the y-axis are factor scores from the first factor (A1) of the CFA model (upper panel) and in the lower panel, factor scores from the general factor of the bifactor model in CFA
None
None
See this image and copyright information in PMC

References

    1. Alderson, C. (2007). The CEFR and the need for more research. The Modern Language Journal, 91, 659- 663. 10.1111/j.1540-4781.2007.00627_4.x - DOI
    2. Alderson C. The CEFR and the need for more research. The Modern Language Journal. 2007;91:659– 663. doi: 10.1111/j.1540-4781.2007.00627_4.x. - DOI
    1. Alexander, G. E., Satalich, T. A., Shankle, W. R., & Batchelder, W. H. (2016). A cognitive psychometric model for the psychodiagnostic assessment of memory-related deficits. Psychological assessment, 28 (3), 279. 10.1037/pas0000163 - DOI - PubMed
    2. Alexander G. E., Satalich T. A., Shankle W. R., Batchelder W. H. A cognitive psychometric model for the psychodiagnostic assessment of memory-related deficits. Psychological assessment. 2016;28(3):279–279. doi: 10.1037/pas0000163. - DOI - PubMed
    1. Asparouhov, T., & Muthén, B. (2009). Exploratory structural equation modeling. Structural Equation Modeling, 16, 397-438. 10.1080/10705510903008204 - DOI
    2. Asparouhov T., Muthén B. Exploratory structural equation modeling. Structural Equation Modeling. 2009;16:397–438. doi: 10.1080/10705510903008204. - DOI
    1. Bonifay, W., & Cai, L. (2017). On the complexity of item response theory models. Multivariate behavioral research, 52 (4), 465-484. 10.1080/00273171.2017.1309262 - DOI - PubMed
    2. Bonifay W., Cai L. On the complexity of item response theory models. Multivariate behavioral research. 2017;52(4):465–484. doi: 10.1080/00273171.2017.1309262. - DOI - PubMed
    1. Bower, J., Runnels, J., Rutson-Griffiths, A., Schmidt, R., Cook, G., Lehde, L., & Kodate, A. (2017). Aligning a Japanese university’s English language curriculum and lesson plans to the CEFR-J. In F. O’Dwyer, M. Hunke, A. Imig, N. Nagai, N. Naganuma, & M. G. Schmidt (Eds.), Critical, Constructive Assessment of CEFR-informed Language Teaching in Japan and Beyond (pp. 176- 225). Cambridge University Press.
    2. Bower J., Runnels J., Rutson-Griffiths A., Schmidt R., Cook G., Lehde L., Kodate A. In: Critical, Constructive Assessment of CEFR-informed Language Teaching in Japan and Beyond. O’Dwyer F., Hunke M., Imig A., Nagai N., Naganuma N., Schmidt M. G., editors. Cambridge University Press; 2017. Aligning a Japanese university’s English language curriculum and lesson plans to the CEFR-J; pp. 176– 225.

LinkOut - more resources