Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2018 Oct 24;20(10):e272.
doi: 10.2196/jmir.9828.

Responsiveness, Reliability, and Minimally Important and Minimal Detectable Changes of 3 Electronic Patient-Reported Outcome Measures for Low Back Pain: Validation Study

Robert Froud  1   2 , Carol Fawkes  3 , Jonathan Foss  1   4 , Martin Underwood  1 , Dawn Carnes  3   5
Affiliations

Responsiveness, Reliability, and Minimally Important and Minimal Detectable Changes of 3 Electronic Patient-Reported Outcome Measures for Low Back Pain: Validation Study

Robert Froud et al. J Med Internet Res. .

Abstract

Background: The Roland Morris Disability Questionnaire (RMDQ), visual analog scale (VAS) of pain intensity, and numerical rating scale (NRS) are among the most commonly used outcome measures in trials of interventions for low back pain. Their use in paper form is well established. Few data are available on the metric properties of electronic counterparts.

Objective: The goal of our research was to establish responsiveness, minimally important change (MIC) thresholds, reliability, and minimal detectable change at a 95% level (MDC95) for electronic versions of the RMDQ, VAS, and NRS as delivered via iOS and Android apps and Web browser.

Methods: We recruited adults with low back pain who visited osteopaths. We invited participants to complete the eRMDQ, eVAS, and eNRS at baseline, 1 week, and 6 weeks along with a health transition question at 1 and 6 weeks. Data from participants reporting recovery were used in MIC and responsiveness analyses using receiver operator characteristic (ROC) curves and areas under the ROC curves (AUCs). Data from participants reporting stability were used for analyses of reliability (intraclass correlation coefficient [ICC] agreement) and MDC95.

Results: We included 442 participants. At 1 and 6 weeks, ROC AUCs were 0.69 (95% CI 0.59 to 0.80) and 0.67 (95% CI 0.46 to 0.87) for the eRMDQ, 0.69 (95% CI 0.58 to 0.80) and 0.74 (95% CI 0.53 to 0.95) for the eVAS, and 0.73 (95% CI 0.66 to 0.80) and 0.81 (95% CI 0.69 to 0.92) for the eNRS, respectively. Associated MIC thresholds were estimated as 1 (0 to 2) and 2 (-1 to 5), 13 (9 to 17) and 7 (-12 to 26), and 2 (1 to 3) and 1 (0 to 2) points, respectively. Over a 1-week period in participants categorized as "stable" and "about the same" using the transition question, ICCs were 0.87 (95% CI 0.66 to 0.95) and 0.84 (95% CI 0.73 to 0.91) for the eRMDQ with MDC95 of 4 and 5, 0.31 (95% CI -0.25 to 0.71) and 0.61 (95% CI 0.36 to 0.77) for the eVAS with MDC95 of 39 and 34, and 0.52 (95% CI 0.14 to 0.77) to 0.67 (95% CI 0.51 to 0.78) with MDC95 of 4 and 3 for the eNRS.

Conclusions: The eRMDQ was reliable with borderline adequate responsiveness. The eNRS was responsive with borderline reliability. While the eVAS had adequate responsiveness, it did not have an attractive reliability profile. Thus, the eNRS might be preferred over the eVAS for measuring pain intensity. The observed electronic outcome measures' metric properties are within the ranges of values reported in the literature for their paper counterparts and are adequate for measuring changes in a low back pain population.

Keywords: Roland Morris Disability Questionnaire; electronic patient-reported outcome measures; minimal detectable change; minimally important change; numerical rating scale; reliability; responsiveness; validation; visual analog scale.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest: RF, MU, and JF are directors and shareholders of Clinvivo Ltd, the University of Warwick spin-out company that provided the software for data collection in this study. The Higher Education Innovation Funding grant paid for the development of intellectual property licensed to Clinvivo and used in this study and also paid for UK retail vouchers used as incentives to recruit participants into the study. RF and DC are nonpracticing osteopaths; CF is a practicing osteopath. MU was chair of the National Institute for Health and Care Excellence accreditation advisory committee, for which he received a fee, until March 2017. MU is chief investigator or co-investigator on multiple previous and current research grants from the UK National Institute for Health Research (NIHR), Arthritis Research UK, Arthritis Australia, and the Australian National Health and Medical Research Council. He has received travel expenses for speaking at conferences from professional organizations hosting the conferences. He is an editor of the NIHR journal series for which he receives a fee. RF and MU have published multiple papers on chronic pain, some of which are referenced in this paper. RF, MU, and JF are part of an academic partnership with Serco Ltd related to return-to-work initiatives.

Figures

Figure 1
Figure 1
Electronic visual analog scale for pain intensity showing 63 units of pain intensity.
Figure 2
Figure 2
Electronic Roland Morris Disability Scale showing a part score of 3 units.
Figure 3
Figure 3
Electronic numerical rating scale for pain intensity showing a part score of 6 units.
Figure 4
Figure 4
Histogram of patient age at baseline.
Figure 5
Figure 5
Flowchart showing completion rates at 1 and 6 weeks, chronicity status, and the incidence of self-reported recovery using the health transition question for participants who also completed the electronic numerical rating scale, and electronic Roland Morris Disability Questionnaire, and electronic visual analog scale measurement.
Figure 6
Figure 6
Graphs showing minimally important change bootstrap standard error convergence from simulations with increasing replication numbers. MIC: minimally important change, NRS: numerical rating scale, RMDQ: Roland Morris Disability Questionnaire, VAS: visual analog scale.
See this image and copyright information in PMC

References

    1. Hartvigsen J, Hancock MJ, Kongsted A, Louw Q, Ferreira ML, Genevay S, Hoy D, Karppinen J, Pransky G, Sieper J, Smeets RJ, Underwood M, Lancet Low Back Pain Series Working Group What low back pain is and why we need to pay attention. Lancet. 2018 Mar 20;:1. doi: 10.1016/S0140-6736(18)30480-X.S0140-6736(18)30480-X - DOI - PubMed
    1. Froud R, Patterson S, Eldridge S, Seale C, Pincus T, Rajendran D, Fossum C, Underwood M. A systematic review and meta-synthesis of the impact of low back pain on people's lives. BMC Musculoskelet Disord. 2014 Feb 21;15:50. doi: 10.1186/1471-2474-15-50. https://www.biomedcentral.com/1471-2474/15/50 1471-2474-15-50 - DOI - PMC - PubMed
    1. Amundsen PA, Evans DW, Rajendran D, Bright P, Bjørkli T, Eldridge S, Buchbinder R, Underwood M, Froud R. Inclusion and exclusion criteria used in non-specific low back pain trials: a review of randomised controlled trials published between 2006 and 2012. BMC Musculoskelet Disord. 2018 Apr 12;19(1):113. doi: 10.1186/s12891-018-2034-6. https://www.biomedcentral.com/1471-2474/19/113 10.1186/s12891-018-2034-6 - DOI - PMC - PubMed
    1. Airaksinen O, Brox JI, Cedraschi C, Hildebrandt J, Klaber-Moffett J, Kovacs F, Mannion AF, Reis S, Staal JB, Ursin H, Zanoli G, COST B13 Working Group on Guidelines for Chronic Low Back Pain European guidelines for the management of chronic nonspecific low back pain. Eur Spine J. 2006 Mar;15 Suppl 2:S192–S300. doi: 10.1007/s00586-006-1072-1. http://europepmc.org/abstract/MED/16550448 - DOI - PMC - PubMed
    1. Maher C, Underwood M, Buchbinder R. Non-specific low back pain. Lancet. 2017 Feb 18;389(10070):736–747. doi: 10.1016/S0140-6736(16)30970-9.S0140-6736(16)30970-9 - DOI - PubMed

Publication types

LinkOut - more resources