Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Dec 3;20(1):293.
doi: 10.1186/s12874-020-01179-5.

COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments: a Delphi study

L B Mokkink  1 , M Boers  2   3 , C P M van der Vleuten  4 , L M Bouter  2   5 , J Alonso  6 , D L Patrick  7 , H C W de Vet  2 , C B Terwee  2
Affiliations

COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments: a Delphi study

L B Mokkink et al. BMC Med Res Methodol. .

Abstract

Background: Scores on an outcome measurement instrument depend on the type and settings of the instrument used, how instructions are given to patients, how professionals administer and score the instrument, etc. The impact of all these sources of variation on scores can be assessed in studies on reliability and measurement error, if properly designed and analyzed. The aim of this study was to develop standards to assess the quality of studies on reliability and measurement error of clinician-reported outcome measurement instruments, performance-based outcome measurement instrument, and laboratory values.

Methods: We conducted a 3-round Delphi study involving 52 panelists.

Results: Consensus was reached on how a comprehensive research question can be deduced from the design of a reliability study to determine how the results of a study inform us about the quality of the outcome measurement instrument at issue. Consensus was reached on components of outcome measurement instruments, i.e. the potential sources of variation. Next, we reached consensus on standards on design requirements (n = 5), standards on preferred statistical methods for reliability (n = 3) and measurement error (n = 2), and their ratings on a four-point scale. There was one term for a component and one rating of one standard on which no consensus was reached, and therefore required a decision by the steering committee.

Conclusion: We developed a tool that enables researchers with and without thorough knowledge on measurement properties to assess the quality of a study on reliability and measurement error of outcome measurement instruments.

Keywords: COSMIN; Delphi study; Measurement error; Outcome measurement instruments; Quality assessment; Reliability; Risk of Bias.

PubMed Disclaimer

Conflict of interest statement

LM, HdV, LB, JA, DP and CT are the developers of the COSMIN Risk of Bias checklist for patient-reported outcome measures (PROMs) [12] and the COSMIN guideline for conducting systematic reviews of PROMs [27].

Figures

Fig. 1
Fig. 1
Content of the Delphi study
See this image and copyright information in PMC

References

    1. Kirshner B, Guyatt G. A methodological framework for assessing health indices. J Chronic Dis. 1985;38(1):27–36. doi: 10.1016/0021-9681(85)90005-0. - DOI - PubMed
    1. Shavelson RJ, Webb NM. A primer. Measurement Methods for the Social Science. Newbury Park: Sage Publishing; 1991. Generalizability Theory.
    1. Brennan RL. Generalizability Theory. New York: Springer; 2001.
    1. Fischer JSJ, Jak AJ, Kniker JE, Rudick RA. Multiple Sclerosis Functional Composite (MSFC): Administration and scoring manual. N.M.S. Society, editor. UNITECH Communications; 2001.
    1. Goodkin DE, Hertsgaard D, Seminary J. Upper extremity function in multiple sclerosis: improving assessment sensitivity with box-and-block and nine-hole peg tests. Arch Phys Med Rehabil. 1988;69(10):850–854. - PubMed

Publication types

LinkOut - more resources