Meta-analysis of test accuracy studies using imputation for partial reporting of multiple thresholds

Abstract

Introduction: For tests reporting continuous results, primary studies usually provide test performance at multiple but often different thresholds. This creates missing data when performing a meta-analysis at each threshold. A standard meta-analysis (no imputation [NI]) ignores such missing data. A single imputation (SI) approach was recently proposed to recover missing threshold results. Here, we propose a new method that performs multiple imputation of the missing threshold results using discrete combinations (MIDC).

Methods: The new MIDC method imputes missing threshold results by randomly selecting from the set of all possible discrete combinations which lie between the results for 2 known bounding thresholds. Imputed and observed results are then synthesised at each threshold. This is repeated multiple times, and the multiple pooled results at each threshold are combined using Rubin's rules to give final estimates. We compared the NI, SI, and MIDC approaches via simulation.

Results: Both imputation methods outperform the NI method in simulations. There was generally little difference in the SI and MIDC methods, but the latter was noticeably better in terms of estimating the between-study variances and generally gave better coverage, due to slightly larger standard errors of pooled estimates. Given selective reporting of thresholds, the imputation methods also reduced bias in the summary receiver operating characteristic curve. Simulations demonstrate the imputation methods rely on an equal threshold spacing assumption. A real example is presented.

Conclusions: The SI and, in particular, MIDC methods can be used to examine the impact of missing threshold results in meta-analysis of test accuracy studies.

Keywords: diagnostic test accuracy; imputation; meta-analysis; multiple thresholds; publication bias.

Figure 1

Mean summary receiver operating characteristic curves for the true test performance across thresholds (top panel), linearity of logit sensitivity across thresholds, and spacing of thresholds across scenarios (bottom panel)

Figure 2

Summary receiver operating characteristic curves—scenarios 7 to 9

Figure 3

Summary receiver operating characteristic curves—scenario 12

Figure 4

Summary receiver operating characteristic curves—scenario 15

Figure 5

Protein/creatinine ratio data—shift in pooled estimates using single imputation (SI) and multiple imputation method based on discrete combinations of missing values (MIDC) vs no imputation (NI)

Figure 6

Standard error of pooled logit sensitivity (top panel), standard error of pooled logit specificity (bottom panel). MIDC, multiple imputation method based on discrete combinations of missing values; NI, no imputation; SI, single imputation