Coverage Evaluation of CTCAE for Capturing the Immune-related Adverse Events Leveraging Text Mining Technologies

Abstract

Background: Immunotherapy is seen as a promising new treatment for cancer but it may also cause immune-related adverse events (irAEs). Post-market surveillance of immunotherapy drugs highly depends on the ability to capture and standardize irAE data. The Common Terminology Criteria for Adverse Events (CTCAE) is a potential terminology that can be leveraged for irAEs standardization. However, the capability of the CTCAE in irAEs standardization needs to be evaluated. Methods: We investigated the irAEs of six FDA approved cancer immunotherapy monoclonal antibodies (mAbs) and evaluated the coverage of the CTCAE for capturing irAEs. We manually identified irAEs from drug labels of the 6 mAbs as the gold standard. We assessed the performance of two text mining pipelines using the dictionary lookup of the CTCAE terms and identified irAEs. In the coverage evaluation, the CTCAE was compared with MedDRA, a standard terminology for regulatory science, for irAE standardization. Results: We manually identified 510 unique irAEs from the drug labels. When using the CTCAE as a dictionary to run the text mining pipeline, the precision, recall and F-measure value was 100%, 10.78% and 19.47%. After adding manually identified irAE terms into the dictionary, the recall and F-measure value significantly improved, increased to 95.69% and 97.31%, respectively. In the coverage evaluation, compared with MedDRA, the coverage rate of the CTCAE is only 13.50% when taking all the mining results together into consideration. Conclusion: With some limitations in our study, we clearly demonstrated that the CTCAE needs an extension to meet the irAE standardization task.

Figure 1.

CTCAE coverage rate for different text mining tasks (compared with MedDRA). 2a is the text mining result for drug labels; 2b is the result for publications; 2c is the result for FAERS-based signal detection; and 2d is the result for systematic evaluation.