Evidence of Misclassification of Drug-Event Associations Classified as Gold Standard 'Negative Controls' by the Observational Medical Outcomes Partnership (OMOP)

Abstract

Introduction: Pharmacovigilance includes analysis of large databases of information on drugs and events using algorithms that detect disproportional frequencies of associations. In order to test such algorithms, attempts have been made to provide canonical reference lists of so-called 'positive controls' and 'negative controls'. Reference sets with even modest levels of misclassification may result in under- or overstatement of the performance of algorithms.

Aim: We sought to determine the extent to which 'negative control' drug-event pairs in the Observational Medical Outcomes Partnership (OMOP) database are misclassified

Methods: We searched the medical literature for evidence of associations between drugs and events listed by OMOP as negative controls.

Results: The criteria used in OMOP to classify positive and negative controls are asymmetric; drug-event associations published only as case series or case reports are classified as positive controls if they are cited in Drug-Induced Diseases by Tisdale and Miller, but as negative controls if case series or case reports exist but are not cited in Tisdale and Miller. Of 233 drug-event pairs classified in the 2013 version of OMOP as negative controls, 21 failed to meet pre-specified OMOP adjudication criteria; in another 19 cases we found case reports, case series, or observational evidence that the drug and event are associated. Overall, OMOP misclassified, or may have misclassified, 40 (17 %) of all 'negative controls.'

Conclusions: Results from studies of the performance of signal-detection algorithms based on the OMOP gold standard should be viewed with circumspection, because imperfect gold standards may lead to under/overstatement of absolute and relative signal detection algorithm performance. Improvements to OMOP would include omitting misclassified drug-event pairs, assigning more specific event labels, and using more extensive sources of information.