Inter-rater reliability of a modified version of Delitto et al.'s classification-based system for low back pain: a pilot study

Abstract

Study design: Observational inter-rater reliability study.

Objectives: To examine: (1) the inter-rater reliability of a modified version of Delitto et al.'s classification-based algorithm for patients with low back pain; (2) the influence of different levels of familiarity with the system; and (3) the inter-rater reliability of algorithm decisions in patients who clearly fit into a subgroup (clear classifications) and those who do not (unclear classifications).

Methods: Patients were examined twice on the same day by two of three participating physical therapists with different levels of familiarity with the system. Patients were classified into one of four classification groups. Raters were blind to the others' classification decision. In order to quantify the inter-rater reliability, percentages of agreement and Cohen's Kappa were calculated.

Results: A total of 36 patients were included (clear classification n = 23; unclear classification n = 13). The overall rate of agreement was 53% and the Kappa value was 0·34 [95% confidence interval (CI): 0·11-0·57], which indicated only fair inter-rater reliability. Inter-rater reliability for patients with a clear classification (agreement 52%, Kappa value 0·29) was not higher than for patients with an unclear classification (agreement 54%, Kappa value 0·33). Familiarity with the system (i.e. trained with written instructions and previous research experience with the algorithm) did not improve the inter-rater reliability.

Conclusion: Our pilot study challenges the inter-rater reliability of the classification procedure in clinical practice. Therefore, more knowledge is needed about factors that affect the inter-rater reliability, in order to improve the clinical applicability of the classification scheme.

Keywords: Agreement; Classification; Low back pain; Physical therapy; Reliability; Subgrouping.

Figure 1

Modified classification-based treatment system for patients with LBP.

Figure 2

Design of the study.

Figure 3

An overview of the inter-rater reliability of Delitto et al.’s classification-based system for low back pain. CI, confidence interval. Inter-rater reliability studies: One examination (raters were allowed to discuss the examination): 1, Apeldoorn et al. Using vignettes: 2, Fritz et al.; 3, Henry et al. Repeated examinations: 4, Fritz and George (no CI available); 5, Heiss et al. (no CI available); 6, Kiesel et al.; 7, Stanton et al.; 8, Apeldoorn et al. (current study).