Artificial Intelligence: Applications in Pharmacovigilance Signal Management

Abstract

Pharmacovigilance is the science of collection, detection, and assessment of adverse events associated with pharmaceutical products for the ongoing monitoring and understanding of those products' safety profiles. Part of this process, signal management, encompasses the activities of signal detection, signal validation/confirmation, signal evaluation, and ultimately, final assessment as to whether a safety signal constitutes a new causal adverse drug reaction. Artificial intelligence is a group of technologies including machine learning and natural language processing that are revolutionizing multiple industries through intelligent automation. Here, we present a critical evaluation of studies leveraging artificial intelligence in signal management to characterize the benefits and limitations of the technology, the level of transparency, and our perspective on best practices for the future. To this end, PubMed and Embase were searched cumulatively for terms pertaining to signal management and artificial intelligence, machine learning, or natural language processing. Information pertaining to the artificial intelligence model used, hyperparameter settings, training/testing data, performance, feature analysis, and more was extracted from included articles. Common signal detection methods included k-means, random forest, and gradient boosting machine. Machine learning algorithms generally outperformed traditional frequentist or Bayesian measures of disproportionality per various metrics, showing the potential utility of advanced machine learning technologies in signal detection. In signal validation and evaluation, natural language processing was typically applied. Overall, methodological transparency was mixed and only some studies leveraged "gold standard" publicly available positive and negative control datasets. Overall, innovation in pharmacovigilance signal management is being driven by machine learning and natural language processing models, particularly in signal detection, in part because of high-performing bagging methods such as random forest and gradient boosting machine. These technologies may be well poised to accelerate progress in this field when used transparently and ethically. Future research is needed to assess the applicability of these techniques across various therapeutic areas and drug classes in the broader pharmaceutical industry.

Fig. 1

Literature search strategy. PubMed and Embase databases were queried for publications pertaining to artificial intelligence (AI) and pharmacovigilance signal management by similar search strategies. Search results were screened first by title for broad relevance, then by abstract for narrow relevance, resulting in 12 final included articles. ML machine learning, NLP natural language processing, PV pharmacovigilance

Fig. 2

Characteristics of included studies. A Included studies by topic (signal detection, signal validation, or signal evaluation). B Included studies by publication year, stratified by topic

Fig. 3

Summary figure. This review summarized publications pertaining to the use of artificial intelligence in signal management. Techniques involved in our 12 included studies were supervised machine learning [support vector machine, random forest, gradient boosting machine, and neural network], natural language processing (GPT, MedLEE), and others. Artificial intelligence performance in signal detection varied, and overall, the number of publications in each topic was heavily biased toward signal detection. AUROC area under the receiver operating characteristic curve