Validation of the Online Collaborative Ocular Tuberculosis Study Calculator for Tubercular Uveitis

Abstract

Importance: This was the first study, to the authors' knowledge, to statistically evaluate the predictive accuracy of Collaborative Ocular Tuberculosis Study (COTS) calculator in guiding initiation of antitubercular therapy (ATT) in patients with clinically suspicious tubercular uveitis (TBU) in an international cohort.

Objective: To evaluate the accuracy of a score of 4 or greater on the online COTS calculator in recommending ATT initiation.

Design, setting, and participants: This study was an evaluation of a diagnostic test or technology. Data input required for the COTS calculator were extracted from the COTS-1 study dataset, which comprised retrospective, observational records of patients with TBU who were monitored for 12 months after treatment. Patients were recruited from international ophthalmic centers. In the absence of a traditional criterion standard, the 12-month treatment response to ATT was used to classify patients as disease positive or negative. The accuracy of clinicians at the ATT decision-making stage in the COTS-1 study was set against COTS calculator scores of 4 or greater. Diagnostic accuracy metrics, including sensitivity, specificity, positive predictive value (PPV), precision, recall, and F1 score, were computed. Data collected from January 2004 to December 2014 were analyzed.

Exposures: COTS calculator to guide initiation of ATT in patients with TBU.

Main outcomes and measures: Comparison of accuracy between clinician judgment and the COTS calculator, analyzed at varying scores and further stratified by tuberculosis endemicity.

Results: Of the 492 participants (mean [SD] age, 42.3 [19.0] years; 233 male [47.3%]), application of the COTS calculator identified 225 (45.7%) with high or very high probability to start ATT (score = 4 or 5) and 111 (22.5%) with very high probability alone (score = 5). COTS-5 exhibited the highest specificity (88.7%; 95% CI, 81.4%-93.8%) compared with clinician judgment (29.6%; 95% CI, 21.4%-38.8%), and clinician judgment led in sensitivity (95.5%; 95% CI, 92.9%-97.4%) compared with COTS-5 (26%; 95% CI, 21.6%-30.7%). COTS-4 and COTS-5 balanced specificity (64.3%; 95% CI, 54.9%-73.1%) and sensitivity (48.8%; 95% CI, 43.7%-54%). PPV and sensitivity were consistently higher in the endemic group for all 3 tests.

Conclusions and relevance: Results of this diagnostic study suggest that the COTS calculator (score ≥4) was more specific than clinician judgment for ATT initiation. Although clinician judgment is a good first step to identify all potential true positives (with high sensitivity), a second consultation with COTS-5 (with high PPV) may lead to less false positives. This tool, apt for high-prevalence, low-resource settings, recommends ATT more selectively for genuine TBU cases. Large prospective studies are essential to explore potential improvements in the calculator's sensitivity.

Figure 1.. Practical Application of the Collaborative Ocular Tuberculosis Study (COTS) Calculator Across 3 Different Scenarios

ATT indicates antitubercular therapy; CT, computed tomography; IGRA, interferon gamma release assay; TB, tuberculosis.

Figure 2.. Potential Outcomes of a Diagnostic Test and Confusion Matrix for Each Test

A, Formula for sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). B, Confusion matrix for the antitubercular therapy (ATT) trial. C, Confusion matrix for Collaborative Ocular Tuberculosis Study (COTS) calculator score 4 or 5. D, Confusion matrix for COTS calculator score 5. FN indicates false negative; FP, false positive; NVP, negative predictive value; TN, true negative; TP, true positive.