Systematic review of prediction models for pulmonary tuberculosis treatment outcomes in adults

Abstract

Objective: To systematically review and critically evaluate prediction models developed to predict tuberculosis (TB) treatment outcomes among adults with pulmonary TB.

Design: Systematic review.

Data sources: PubMed, Embase, Web of Science and Google Scholar were searched for studies published from 1 January 1995 to 9 January 2020.

Study selection and data extraction: Studies that developed a model to predict pulmonary TB treatment outcomes were included. Study screening, data extraction and quality assessment were conducted independently by two reviewers. Study quality was evaluated using the Prediction model Risk Of Bias Assessment Tool. Data were synthesised with narrative review and in tables and figures.

Results: 14 739 articles were identified, 536 underwent full-text review and 33 studies presenting 37 prediction models were included. Model outcomes included death (n=16, 43%), treatment failure (n=6, 16%), default (n=6, 16%) or a composite outcome (n=9, 25%). Most models (n=30, 81%) measured discrimination (median c-statistic=0.75; IQR: 0.68-0.84), and 17 (46%) reported calibration, often the Hosmer-Lemeshow test (n=13). Nineteen (51%) models were internally validated, and six (16%) were externally validated. Eighteen (54%) studies mentioned missing data, and of those, half (n=9) used complete case analysis. The most common predictors included age, sex, extrapulmonary TB, body mass index, chest X-ray results, previous TB and HIV. Risk of bias varied across studies, but all studies had high risk of bias in their analysis.

Conclusions: TB outcome prediction models are heterogeneous with disparate outcome definitions, predictors and methodology. We do not recommend applying any in clinical settings without external validation, and encourage future researchers adhere to guidelines for developing and reporting of prediction models.

Trial registration: The study was registered on the international prospective register of systematic reviews PROSPERO (CRD42020155782).

Keywords: epidemiology; statistics & research methods; tuberculosis.

Figure 1

PRISMA flow chart of inclusion process. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses

Figure 2

Most common predictors considered and included. Considered: the predictor as evaluated as a candidate predictor prior to multivariable modelling. Included: the predictor was considered and subsequently included in the final multivariable model. BMI, body mass index; MDR, multidrug resistant; TB, tuberculosis.

Figure 3

Heatmap of signalling questions from risk of bias assessment with PROBAST. PROBAST questions (additional details in online supplemental file 5) Participants 1: what study design was used and was it appropriate? Participants 2: were all inclusion and exclusion criteria appropriate? Predictors 1: were predictors defined as assessed the same way for all participants? Predictors 2: were predictor assessments made without knowledge of data outcome? Predictors 3: are all predictors available at the time the model was intended to be used? Outcome 1: was the outcome determined appropriately? Outcome 2: was the outcome pre-specified or standard? Outcome 3: were predictors excluded from outcome definition? Outcome 4: was the outcome defined and determined in a similar way for all participants? Outcome 5: was the outcome determined without predictor information? Outcome 6: was the time interval between predictor assessment and outcome determination appropriate? Analysis 1: were there a reasonable number of participants with the outcome? Analysis 2: were continuous and categorical variables handled appropriately? Analysis 3: were all enroled participants included in the analysis? Analysis 4: were participants with missing data handled appropriately? Analysis 5: was selection of predictors based on univariable analysis avoided? Analysis 6: were complexities in data (censoring, competing risks, sampling of control participants) accounted for appropriately? Analysis 7: were relevant model performance measures evaluated appropriately? Analysis 8: were model overfitting, underfitting, and optimism in the model performance accounted for? Analysis 9: do predictors and their assigned weights in the final model correspond to the results from the reported multivariable analysis?.

Figure 4

Summary of risk of bias and applicability assessment with PROBAST. PROBAST, Prediction Model Risk of Bias Assessment Tool.