A retrospective evaluation of a data mining approach to aid finding new adverse drug reaction signals in the WHO international database

Abstract

Background: The detection of new drug safety signals is of growing importance with ever more new drugs becoming available and exposure to medicines increasing. The task of evaluating information relating to safety lies with national agencies and, for international data, with the World Health Organization Programme for International Drug Monitoring.

Rationale: An established approach for identifying new drug safety signals from the international database of more than 2 million case reports depends upon clinical experts from around the world. With a very large amount of information to evaluate, such an approach is open to human error. To aid the clinical review, we have developed a new signalling process using Bayesian logic, applied to data mining, within a confidence propagation neural network (Bayesian Confidence Propagation Neural Network; BCPNN). Ultimately, this will also allow the evaluation of complex variables.

Methods: The first part of this study tested the predictive value of the BCPNN in new signal detection as compared with reference literature sources (Martindale's Extra Pharmacopoeia in 1993 and July 2000, and the Physicians Desk Reference in July 2000). In the second part of the study, results with the BCPNN method were compared with those of the former signalling procedure.

Results: In the study period (the first quarter of 1993) 107 drug-adverse reaction combinations were highlighted as new positive associations by the BCPNN, and referred to new drugs. 15 drug-adverse reaction combinations on new drugs became negative BCPNN associations in the study period. The BCPNN method detected signals with a positive predictive value of 44% and the negative predictive value was 85%. 17 as yet unconfirmed positive associations could not be dismissed with certainty as false positive signals. Of the 10 drug-adverse reaction signals produced by the former signal detection system from data sent out for review during the study period, 6 were also identified by the BCPNN. These 6 associations have all had a more than 10-fold increase of reports and 4 of them have been included in the reference sources. The remaining 4 signals that were not identified by the BCPNN had a small, or no, increase in the number of reports, and are not listed in the reference sources.

Conclusion: Our evaluation showed that the BCPNN approach had a high and promising predictive value in identifying early signals of new adverse drug reactions.