A Prediction Model to Identify Clinically Relevant Medication Discrepancies at the Emergency Department (MED-REC Predictor): Development and Validation Study

Abstract

Background: Many patients do not receive a comprehensive medication reconciliation, mostly owing to limited resources. We hence need an approach to identify those patients at the emergency department (ED) who are at increased risk for clinically relevant discrepancies.

Objective: The aim of our study was to develop and externally validate a prediction model to identify patients at risk for at least 1 clinically relevant medication discrepancy upon ED presentation.

Methods: A prospective, multicenter, observational study was conducted at the University Hospitals Leuven and General Hospital Sint-Jan Brugge-Oostende AV, Belgium. Medication histories were obtained from patients admitted to the ED between November 2017 and May 2022, and clinically relevant medication discrepancies were identified. Three distinct datasets were created for model development, temporal external validation, and geographic external validation. Multivariable logistic regression with backward stepwise selection was used to select the final model. The presence of at least 1 clinically relevant discrepancy was the dependent variable. The model was evaluated by measuring calibration, discrimination, classification, and net benefit.

Results: We included 824, 350, and 119 patients in the development, temporal validation, and geographic validation dataset, respectively. The final model contained 8 predictors, for example, age, residence before admission, number of drugs, and number of drugs of certain drug classes based on Anatomical Therapeutic Chemical coding. Temporal validation showed excellent calibration with a slope of 1.09 and an intercept of 0.18. Discrimination was moderate with a c-index (concordance index) of 0.67 (95% CI 0.61-0.73). In the geographic validation dataset, the calibration slope and intercept were 1.35 and 0.83, respectively, and the c-index was 0.68 (95% CI 0.58-0.78). The model showed net benefit over a range of clinically reasonable threshold probabilities and outperformed other selection criteria.

Conclusions: Our software-implemented prediction model shows moderate performance, outperforming random or typical selection criteria for medication reconciliation. Depending on available resources, the probability threshold can be customized to increase either the specificity or the sensitivity of the model.

Keywords: MED-REC; MED-REC predictor; emergency department; geographic; geographic validation; hospital; medication; medication discrepancy; medication reconciliation; patient; prediction model; predictor; risk stratification; software; software-implemented prediction model.

Figure 1

Study flow diagram.

Figure 2

Calibration plots for the MED-REC predictor in the (A) development, (B) temporal validation, and (C) and geographic validation dataset, as well as AUROC curves for the MED-REC predictor in the (D) development, (E) temporal validation, (F) and geographic validation dataset. The red line represents perfect calibration. The shaded area represents the 95% CI. AUROC: area under the receiver operating characteristic curve.

Figure 3

Calibration plot for the updated MED-REC predictor in the geographic validation dataset. The red line represents perfect calibration. The shaded area represents the 95% CI.

Figure 4

Decision curve analysis for the MED-REC predictor in the development dataset. Red line (treat all): all patients receive medication reconciliation. Green line (treat none): none of the patients receives medication reconciliation. Blue line: selection of patients who are ≥75 years and take ≥5 drugs. Purple line: MED-REC predictor. The MED-REC predictor shows net benefit over the range of clinically reasonable threshold probabilities. Classifying a patient as having at least 1 clinically relevant medication discrepancy will trigger performing a comprehensive medication reconciliation. Accordingly, lower threshold probabilities are preferred if one is worried about missing clinically relevant discrepancies. Higher threshold probabilities are preferred if one is worried about additional costs or resources associated with medication reconciliation. For instance, a probability threshold of 20% implies that you expect to find at least 1 clinically relevant discrepancy in 1 out of 5 reconciled patients.

Figure 5

Comparison of the (A) MED-REC predictor to (B) random selection. Calculations are based on the classification measures of the temporal validation. A probability threshold of 0.45 was used, considering limited staffing resources for medication reconciliation in our hospitals. Patients with at least 1 clinically relevant discrepancy are presented in yellow. Patients without clinically relevant discrepancies are presented in white. Medication reconciliation is performed for patients in green. (A) If the MED-REC predictor is applied to 100 patients, 17 will be identified as high risk, alerting the pharmacist to perform a medication reconciliation. For 9 of these patients (17×0.55), at least 1 clinically relevant medication discrepancy will be found. (B) When a medication reconciliation is performed for 17 randomly selected patients, at least 1 clinically relevant medication discrepancy will be found in 6 patients (17×0.37). Eleven patients, as opposed to 8 with the MED-REC predictor, would not have needed the pharmacy staff’s intervention.