A Systematic Review of Propensity Score Matching in the Orthopedic Literature

Abstract

Background: Propensity score matching (PSM) is a statistical technique used to reduce bias in observational studies by controlling for measured confounders. Given its complexity and popularity, it is imperative that researchers comprehensively report their methodologies to ensure accurate interpretation and reproducibility.

Purpose: This systematic review sought to define how often PSM has been used in recent orthopedic research and to describe how such studies reported their methods. Secondary aims included analyzing study reproducibility, bibliometric factors associated with reproducibility, and associations between methodology and the reporting of statistically significant results.

Methods: PubMed and Embase databases were queried for studies containing "propensity score" and "match*" published in 20 orthopedic journals prior to 2020. All studies meeting inclusion criteria were used for trend analysis. Articles published between 2017 and 2019 were used for analysis of reporting quality and reproducibility.

Results: In all, 261 studies were included for trend analysis, and 162 studies underwent full-text review. The proportion of orthopedic studies using PSM significantly increased over time. Seventy-one (41%) articles did not provide justification for covariate selection. The majority of studies illustrated covariate balance through P values. We found that 19% of the studies were fully reproducible. Most studies failed to report the use of replacement (67.3%) or independent or paired statistical methods (34.0%). Studies reporting standardized mean differences to illustrate covariate balance were less likely to report statistically significant results.

Conclusion: Despite the increased use of PSM in orthopedic research, observational studies employing PSM have largely failed to adequately report their methodology.

Keywords: biostatistics; observational studies; propensity score matching; reproducibility; scientific reporting.

Fig. 1.

A flow diagram of the search strategy. PSM Propensity Score Matching.

Fig. 2.

Trend analysis depicting the absolute and relative increase in orthopedic observational studies employing PSM over time. PSM Propensity Score Matching.