The Missing Medians: Exclusion of Ordinal Data from Meta-Analyses

Abstract

Background: Meta-analyses are considered the gold standard of evidence-based health care, and are used to guide clinical decisions and health policy. A major limitation of current meta-analysis techniques is their inability to pool ordinal data. Our objectives were to determine the extent of this problem in the context of neurological rating scales and to provide a solution.

Methods: Using an existing database of clinical trials of oral neuroprotective therapies, we identified the 6 most commonly used clinical rating scales and recorded how data from these scales were reported and analysed. We then identified systematic reviews of studies that used these scales (via the Cochrane database) and recorded the meta-analytic techniques used. Finally, we identified a statistical technique for calculating a common language effect size measure for ordinal data.

Results: We identified 103 studies, with 128 instances of the 6 clinical scales being reported. The majority- 80%-reported means alone for central tendency, with only 13% reporting medians. In analysis, 40% of studies used parametric statistics alone, 34% of studies employed non-parametric analysis, and 26% did not include or specify analysis. Of the 60 systematic reviews identified that included meta-analysis, 88% used mean difference and 22% employed difference in proportions; none included rank-based analysis. We propose the use of a rank-based generalised odds ratio (WMW GenOR) as an assumption-free effect size measure that is easy to compute and can be readily combined in meta-analysis.

Conclusion: There is wide scope for improvement in the reporting and analysis of ordinal data in the literature. We hope that adoption of the WMW GenOR will have the dual effect of improving the reporting of data in individual studies while also increasing the inclusivity (and therefore validity) of meta-analyses.

Fig 1. Reporting of central tendency for the 6 clinical rating scales.

Fig 2. Approach to statistical analysis for the 6 clinical rating scales.

Fig 3. Approach to pooling data from the 6 clinical rating scales in systematic reviews.

Fig 4. Histograms of mock EDSS data in a control group (sample 0) and 5 treatment groups (samples 1–5).

Fig 5. Random effects meta-analysis, based on standardised mean difference, of 5 simulated studies using mock EDSS data.

Fig 6. Ordinal meta-analysis, based on WMW GenOR, of 5 simulated studies using mock EDSS data.

Fig 7. Ordinal meta-analysis, based on WMW GenOR calculated from summary statistics, of 5 simulated studies using mock EDSS data.