Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2024 Oct;33(10):2593-2609.
doi: 10.1007/s11136-024-03706-z. Epub 2024 Jul 3.

Methodological quality of 100 recent systematic reviews of health-related outcome measurement instruments: an overview of reviews

Ellen B M Elsman  1   2 , Lidwine B Mokkink  1 , Inger L Abma  3 , Olalekan Lee Aiyegbusi  4 , Alessandro Chiarotto  5 , Kirstie L Haywood  6 , Karen Matvienko-Sikar  7 , Daniella M Oosterveer  8 , Jan J M Pool  9 , Ilse E J Swinkels-Meewisse  10 , Martin Offringa  2 , Caroline B Terwee  11
Affiliations
Review

Methodological quality of 100 recent systematic reviews of health-related outcome measurement instruments: an overview of reviews

Ellen B M Elsman et al. Qual Life Res. 2024 Oct.

Abstract

Purpose: Systematic reviews evaluating and comparing the measurement properties of outcome measurement instruments (OMIs) play an important role in OMI selection. Earlier overviews of review quality (2007, 2014) evidenced substantial concerns with regards to alignment to scientific standards. This overview aimed to investigate whether the quality of recent systematic reviews of OMIs lives up to the current scientific standards.

Methods: One hundred systematic reviews of OMIs published from June 1, 2021 onwards were randomly selected through a systematic literature search performed on March 17, 2022 in MEDLINE and EMBASE. The quality of systematic reviews was appraised by two independent reviewers. An updated data extraction form was informed by the earlier studies, and results were compared to these earlier studies' findings.

Results: A quarter of the reviews had an unclear research question or aim, and in 22% of the reviews the search strategy did not match the aim. Half of the reviews had an incomprehensive search strategy, because relevant search terms were not included. In 63% of the reviews (compared to 41% in 2014 and 30% in 2007) a risk of bias assessment was conducted. In 73% of the reviews (some) measurement properties were evaluated (58% in 2014 and 55% in 2007). In 60% of the reviews the data were (partly) synthesized (42% in 2014 and 7% in 2007); evaluation of measurement properties and data syntheses was not conducted separately for subscales in the majority. Certainty assessments of the quality of the total body of evidence were conducted in only 33% of reviews (not assessed in 2014 and 2007). The majority (58%) did not make any recommendations on which OMI (not) to use.

Conclusion: Despite clear improvements in risk of bias assessments, measurement property evaluation and data synthesis, specifying the research question, conducting the search strategy and performing a certainty assessment remain poor. To ensure that systematic reviews of OMIs meet current scientific standards, more consistent conduct and reporting of systematic reviews of OMIs is needed.

Keywords: COSMIN; Measurement properties; Outcome measurement instruments; Reliability; Systematic reviews; Validity.

PubMed Disclaimer

Conflict of interest statement

The authors have no relevant financial or non-financial interests to disclose. Drs. Terwee and Mokkink are the founders of COSMIN.

Figures

Fig. 1
Fig. 1
COSMIN tools and methodology
See this image and copyright information in PMC

References

    1. Devlin, N. J., & Appleby, J. (2010). Getting the most out of PROMs. Putting health outcomes at the heart of NHS decision-making. The King’s Fund.
    1. Fitzpatrick, R., Davey, C., Buxton, M. J., & Jones, D. R. (1998). Evaluating patient-based outcome measures for use in clinical trials. Health Technology Assessment. 10.3310/hta2140 - PubMed
    1. Greenhalgh, J. (2009). The applications of PROs in clinical practice: What are they, do they work, and why? Quality of Life Research,18, 115–123. - PubMed
    1. Prinsen, C. A., Mokkink, L. B., Bouter, L. M., Alonso, J., Patrick, D. L., De Vet, H. C., & Terwee, C. B. (2018). COSMIN guideline for systematic reviews of patient-reported outcome measures. Quality of Life Research,27(5), 1147–1157. - PMC - PubMed
    1. Mokkink, L. B., Prinsen, C. A., Bouter, L. M., de Vet, H. C., & Terwee, C. B. (2016). The COnsensus-based standards for the selection of health measurement instruments (COSMIN) and how to select an outcome measurement instrument. Brazilian Journal of Physical Therapy,20, 105–113. - PMC - PubMed

LinkOut - more resources