Knowledge of Problem Solving (KOPS) Scale: Design and Evaluation of a Digitally Administered Competence Measure for a Common Practice Element in Task-Shared Youth Mental Health Interventions

Abstract

There are few reliable and feasible quality assurance methods to support scaling up of psychological interventions delivered by non-specialist providers. This paper reports on the phased development and validation of a digitally administered Knowledge of Problem Solving (KOPS) measure to assess competencies associated with a "task-shared" problem-solving intervention for adolescents with diverse mental health problems in India. Phase 1 established key competencies required to deliver the intervention, followed by item generation for a corresponding knowledge-based competency measure that could be administered efficiently through e-learning systems. In phase 2, items were refined based on responses from an "experienced" reference sample comprising 17 existing counsellors with direct experience of the problem-solving intervention, and a "novice" sample with 14 untrained university students and NGO staff. In phase 3, we evaluated two parallel versions of the measure in a validation sample (N = 277) drawn from universities and NGOs. The resulting 17-item measure was structured around a hypothetical case, followed by multiple-choice questions that asked about the most appropriate response to a practice-based scenario. The difficulty level of the test items was well matched to the ability level of participants (i.e. most items being of moderate difficulty and few items being easy or difficult). Only one item showed a negative discrimination index and was removed from the 17-item forms. The final 16-item version of the KOPS measure provides a scalable digital method to assess key psychotherapeutic competencies among non-specialists, particularly in relation to a transdiagnostic problem-solving intervention. Similar formats could be deployed more widely alongside e-learning programmes to expand the global workforce capable of delivering evidence-based psychological interventions.

Supplementary information: The online version contains supplementary material available at 10.1007/s41347-023-00356-9.

Keywords: Adolescent mental health; Capacity building; Digital technology; India; Measure development; Problem-solving intervention; Psychometrics.

Fig. 1

Item characteristics curve for Form A

Fig. 2

Item characteristics curve for Form B