Large-scale combining signals from both biomedical literature and the FDA Adverse Event Reporting System (FAERS) to improve post-marketing drug safety signal detection

Abstract

Background: Independent data sources can be used to augment post-marketing drug safety signal detection. The vast amount of publicly available biomedical literature contains rich side effect information for drugs at all clinical stages. In this study, we present a large-scale signal boosting approach that combines over 4 million records in the US Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS) and over 21 million biomedical articles.

Results: The datasets are comprised of 4,285,097 records from FAERS and 21,354,075 MEDLINE articles. We first extracted all drug-side effect (SE) pairs from FAERS. Our study implemented a total of seven signal ranking algorithms. We then compared these different ranking algorithms before and after they were boosted with signals from MEDLINE sentences or abstracts. Finally, we manually curated all drug-cardiovascular (CV) pairs that appeared in both data sources and investigated whether our approach can detect many true signals that have not been included in FDA drug labels. We extracted a total of 2,787,797 drug-SE pairs from FAERS with a low initial precision of 0.025. The ranking algorithm combined signals from both FAERS and MEDLINE, significantly improving the precision from 0.025 to 0.371 for top-ranked pairs, representing a 13.8 fold elevation in precision. We showed by manual curation that drug-SE pairs that appeared in both data sources were highly enriched with true signals, many of which have not yet been included in FDA drug labels.

Conclusions: We have developed an efficient and effective drug safety signal ranking and strengthening approach We demonstrate that large-scale combining information from FAERS and biomedical literature can significantly contribute to drug safety surveillance.

Figure 1

Data and experimental flowchart. The two large data sources for drug-SE extraction are 4,285,094 records from FAERS and 21,354,075 MEDLINE records. The process included: (1) drug-SE pair extraction from FAERS; (2) Ranking extracted pairs using six commonly used statistical signal detection approaches, and boosting the rankings by pairs’ MEDLINE presence; and (3) manual curation of all targeted anticancer drug associated cardiovascular events that appeared in both FAERS and MEDLINE and compared them to those captured in FDA drug labeling.

Figure 2

Precision-recall curves of ranked drug-SE pairs. The ranked precision-recall curves for pairs ranked by FAERS signals (“FREQ”, “PRR”, “OffSides”) alone, and ranked by FAERS signals augmented by pairs’ presence in MEDLINE (“FREQ_boosted_sentence”, “FREQ_boosted_abstract”, “PRR_boosted_sentence”, “PRR_boosted_abstract”, “OffSides_boosted_sentence”, “OffSides_boosted_abstract”). Rankings by RRR, ROR, IC and PhiCorr had similar performance as that of ranking by PRR (data not shown).