Making a case for a change to using CEFR-oriented placement test scores: A reflexive ethnographic decision making

Abstract

The term 'language testing and assessment' (LTA) seems to often invoke ideas related to numbers, such as mathematical equation and statistical calculation. This may range from scoring performance or responses to examination questions as numbers, to informing stakeholders of test results. This article investigates a series of actions pertaining to LTA by bringing in reflexive ethnography (Davies, C.A. [2008]. Reflexive Ethnography: A Guide to Researching Selves and Others. Routledge: London.). The aims are to add a humanistic element to the discipline in general and to feed another piece of applicability information to ethnographers in particular. Attempting at introducing such an approach largely qualitative as ethnography to the field of LTA can be useful because it may open up possibilities for balanced views in assessing language competence in future studies, in which social factors can also be taken explicitly into account for a decision making in LTA. For ethnographers, applicability of reflexive ethnography in LTA could mean the methodology achieves greater generalizability through a larger sample in the universe of admissible observations. A primary research question is whether a change in using CEFR-oriented placement test scores can be justified using empirical reasoning. Participatory observations, situational weigh-in, and statistical analyses are found capable of synergizing using a reflexive-ethnographic perspective for a justifiable decision-making process.

Keywords: Applied linguistics; Education; Ethnography; Justification; Language assessment; Linguistics; Placement test; Program evaluation; Reflexive ethnography; Sociology.

Figure 1

Investigation framework of the article.

Figure 2

Re-profiling plan for the university administration.

Figure 3

A hierarchy of requirements for an educational change.

Figure 4

Branding walls and memorabilia for the re-profiling policy: (clockwise) 1. a cap; 2. an organizer; 3.–4. branding walls; 5. a shopping bag; 6. a calendar.

Figure 5

Number of students starting the first year across CEFR levels.

Figure 6

Centripetal force of assessment.

Figure 7

Chain of assessment events for adjusting an exemption program.

Figure 8

Correlation of scores from the CEFR-based placement test and English III: 1. correlation in Term 1 of the 2017 academic year; 2. correlation in Term 2 of the 2017 academic year.

Figure 9

Drive of assessment for decision making.