Reliability of Identification of Behavior Change Techniques in Intervention Descriptions

Abstract

Purpose: The aim of this paper is to assess the frequency of identification as well as the inter-coder and test-retest reliability of identification of behavior change techniques (BCTs) in written intervention descriptions.

Methods: Forty trained coders applied the "Behavior Change Technique Taxonomy version 1" (BCTTv1) to 40 intervention descriptions published in protocols and repeated this 1 month later.

Results: Eighty of 93 defined BCTs were identified by at least one trained coder, and 22 BCTs were identified in 16 (40 %) or more of 40 descriptions. Good inter-coder reliability was observed across 80 BCTs identified in the protocols: 66 (80 %) achieved mean prevalence and bias-adjusted kappa (PABAK) scores of 0.70 or greater, and 59 (74 %) achieved mean scores of 0.80 or greater. There was good within-coder agreement between baseline and 1 month, demonstrating good test-retest reliability.

Conclusions: BCTTv1 can be used by trained coders to identify BCTs in intervention descriptions reliably. However, some frequently occurring BCT definitions require further clarification.

Keywords: Behavior change; Behavior change technique; Inter-coder reliability; Intervention; Taxonomy; Test–retest reliability.