An end-to-end hybrid algorithm for automated medication discrepancy detection

Abstract

Background: In this study we implemented and developed state-of-the-art machine learning (ML) and natural language processing (NLP) technologies and built a computerized algorithm for medication reconciliation. Our specific aims are: (1) to develop a computerized algorithm for medication discrepancy detection between patients' discharge prescriptions (structured data) and medications documented in free-text clinical notes (unstructured data); and (2) to assess the performance of the algorithm on real-world medication reconciliation data.

Methods: We collected clinical notes and discharge prescription lists for all 271 patients enrolled in the Complex Care Medical Home Program at Cincinnati Children's Hospital Medical Center between 1/1/2010 and 12/31/2013. A double-annotated, gold-standard set of medication reconciliation data was created for this collection. We then developed a hybrid algorithm consisting of three processes: (1) a ML algorithm to identify medication entities from clinical notes, (2) a rule-based method to link medication names with their attributes, and (3) a NLP-based, hybrid approach to match medications with structured prescriptions in order to detect medication discrepancies. The performance was validated on the gold-standard medication reconciliation data, where precision (P), recall (R), F-value (F) and workload were assessed.

Results: The hybrid algorithm achieved 95.0%/91.6%/93.3% of P/R/F on medication entity detection and 98.7%/99.4%/99.1% of P/R/F on attribute linkage. The medication matching achieved 92.4%/90.7%/91.5% (P/R/F) on identifying matched medications in the gold-standard and 88.6%/82.5%/85.5% (P/R/F) on discrepant medications. By combining all processes, the algorithm achieved 92.4%/90.7%/91.5% (P/R/F) and 71.5%/65.2%/68.2% (P/R/F) on identifying the matched and the discrepant medications, respectively. The error analysis on algorithm outputs identified challenges to be addressed in order to improve medication discrepancy detection.

Conclusion: By leveraging ML and NLP technologies, an end-to-end, computerized algorithm achieves promising outcome in reconciling medications between clinical notes and discharge prescriptions.

Figure 1

The example overview note (a), discharge summary (b) and discharge prescription list (c) for an encounter. The medication information identified by the annotators is highlighted in clinical notes.

Figure 2

The architecture of the proposed automated medication discrepancy detection algorithm. Bullet 1–3 represent the three algorithm processes. Bullet A-C represent the outputs of the processes that were evaluated in the study. *“Matched medication list” includes the medication entities mentioned in the clinical notes and that had matches in the prescription list. **“Discrepant medication list” includes the medication entities mentioned in the clinical notes but were missed in the prescription list, or had medication attributes that were not consistent between the notes and the list.

Figure 3

The diagram of the medication matching process.

Figure 4

The overall inter-annotator agreements (IAAs; F-value) for overview notes and discharge summaries (a). The IAAs on individual entity categories are also presented (b and c).

Figure 5

The recalls of medication name detection on discrepant medications.