Meta-analysis to compare the accuracy of GeneXpert, MODS and the WHO 2007 algorithm for diagnosis of smear-negative pulmonary tuberculosis

Abstract

Background: Smear-negative pulmonary tuberculosis (SN-PTB), which is common in HIV-infected patients, is difficult to diagnose using smear microscopy alone. In 2007, the WHO developed an algorithm to improve the diagnosis and management of smear-negative tuberculosis in HIV prevalent and resource constrained settings. Implementation of the algorithm required individuals with presumptive TB to be initially evaluated using two sputum microscopy examinations followed by clinical diagnosis that may include chest X-ray and antibiotic treatment in smear-negative individuals. Since that time, the WHO has endorsed several new tests for diagnosis of tuberculosis. However, it is unclear how the new tests perform when compared to the WHO 2007 algorithm in diagnosis of SN-PTB. Using meta-analysis study design, we summarized and compared the accuracy of Xpert® MTB/Rif assay (GeneXpert) and Microscopic Observation Drug Susceptibility assay (MODS), with the WHO 2007 algorithm in the diagnosis of SN-PTB.

Methods: A systematic review and meta-analysis of publications on GeneXpert, or MODS, or the WHO 2007 algorithm for diagnosis of SN-PTB, using culture as reference test was performed. Meta-Disc software was used to obtain pooled sensitivity and specificity of the diagnostic methods. Heterogeneity in the accuracy estimates was tested by reviewing the generated forest plots, sROC curves and the Spearman correlation coefficient of the logit of true positive rate versus the logit of false positive rate.

Results: Twenty-four publications on all three diagnostic methods were meta-analyzed. The pooled sensitivity and specificity for detection of smear-negative pulmonary tuberculosis were 67% and 98% for GeneXpert, 73% and 91% for MODS, and 61% and 69% for WHO 2007 algorithm, respectively. The sensitivity of GeneXpert reduced from 67% to 54% when sub-group analysis of studies with patient HIV prevalence ≥ 30% was performed.

Conclusion: The GeneXpert, MODS, and the WHO algorithm have moderate to high accuracy for the diagnosis of SN-PTB. However, the accuracy of the tests is extremely variable. The setting and context under which the tests are conducted in addition to several other factors could explain this variability. There is therefore need to investigate these factors further. The information from these studies would inform the adoption and placement of these new tests.

Figure 1

WHO 2007 algorithm for the diagnosis of TB in ambulatory HIV-positive patients. a) the danger signs include any one of: respiratory rate > 30/minute, fever > 39°C, pulse rate > 120/min and unable to walk unaided. b) for countries with adult HIV prevalence rate = 1% or prevalence rate of HIV among tuberculosis patients = 5%. c) In the absence of HIV testing, classifying HIV status unknown as HIV-positive depends on clinical assessment or national and/or local policy. d) AFB-positive is defined at least one positive and AFB-negative as two or more negative smears. e) CPT = Co-trimoxazole preventive therapy. f) HIV assessment includes HIV clinical staging, determination of CD count if available and referral for HIV care. g) the investigations within the box should be done at the same time wherever possible in order to decrease the number of visits and speed up the diagnosis. h) antibiotics (except fluoroquinolones) to cover both typical and atypical bacteria should be considered. i) PCP: Pneumocystis carinii pneumonia, also known as Pneumocystis jirovecii pneumonia. j) advise to return for reassessment if symptoms recur.

Figure 2

Flow chart for publication search. * One of the WHO publications provided separate diagnostic accuracy results for a rural and urban site. The results were therefore reported separately in Table 1. SNPTB = Smear-negative Pulmonary TB. Two landmark studies on GeneXpert were excluded [22,23]. We contacted the author but the data provided remained incomplete to fill 2x2 tables for smear-negative PTB (SNPTB).

Figure 3

Forest plots of sensitivity and specificity for (a) GeneXpert test, (b) MODS test and (c) WHO 2007 algorithm.

Figure 4

Summary receiver characteristics (sROC) (a) curve- GeneXpert, (b) curve- MODS and (c) curve- WHO 2007 algorithm.Note: sROC = summary receiver operating characteristic curve, which is a plot of the true positive rate (sensitivity) against the false positive rate (1-specificity) of a diagnostic test at different thresholds [47]. This generates a composite statistic (AUC or the Index Q*) that provides an overall evaluation of the accuracy of a test (perfect discriminating ability of true positivity from false positivity). The three curves of the sROC represent the estimate and the 95% upper and lower bounds of the estimate. AUC = Area under the curve of a constructed sROC curve. An AUC close to 1.0 signifies that the test has almost perfect discrimination while an AUC close to 0.5 suggests poor discrimination. An AUC significantly less than 0.5 would indicate that the criteria for “normal” and “abnormal” should be reversed. SE (AUC) = standard error of the area under curve Q* = An index which corresponds to the upper most point on the sROC curve at which sensitivity equals specificity. The closer this value is to 1, the closer the test to perfect accuracy (perfect discriminating ability of true positivity from false positivity). When the value of the Q* index is close to 0.5, it signifies that the test has poor discrimination. SE (Q*) = the standard error of the index Q*.

Figure 5

Forest plots of sub-analysis of sensitivity and specificity of GeneXpert.

Figure 6

QUADAS-2 Results of (a) risk of bias and (b) concerns on applicability.