Development of the treatment prediction model in the artificial intelligence in depression - medication enhancement study

Abstract

We introduce an artificial intelligence model to personalize treatment in major depression, which was deployed in the Artificial Intelligence in Depression: Medication Enhancement Study. We predict probabilities of remission across multiple pharmacological treatments, validate model predictions, and examine them for biases. Data from 9042 adults with moderate to severe major depression from antidepressant clinical trials were used to train a deep learning model. On the held-out test-set, the model demonstrated an AUC of 0.65, outperformed a null model (p = 0.01). The model increased population remission rate in hypothetical and actual improvement testing. While the model identified escitalopram as generally outperforming other drugs (consistent with the input data), there was otherwise significant variation in drug rankings. The model did not amplify potentially harmful biases. We demonstrate the first model capable of predicting outcomes for 10 treatments, intended to be used at or near the start of treatment to personalize treatment selection.

Fig. 1

PRISMA diagram for study inclusion and included medications.

Fig. 2. Test Set Calibration Plots.

Calibration plot for the test set. ECE expected calibration error, MCE maximum calibration error.

Fig. 3. Bias Plots.

Plots of observed and predicted remission probability for drug, sex, ethnicity, and age groups (extreme right). Error bars represent standard error. A shows result for the training set; B for the validation set; and C for the test set.

Fig. 4. Selected Sensitivity Plots- Test Set.

Plots of remission rate predicted by the model as each variable (suicidal ideation in (A) and baseline weight loss in (B) at baseline is increased.

Fig. 5. Example output from the Clinical Decision Support System.

Top: Example result for (A) escitalopram and (B) duloxetine prediction. Information displayed includes drug class, typical dose, effective dose range, minimum/maximum dose range, dosing tips, clinical pearls, patient-specific remission probability for escitalopram, patient-specific remission probability across treatments, and population baseline. [Note: Duloxetine is an SNRI; this was a known error which was intended to be fixed in a software patch to be done after this screenshot was taken]. Bottom: Example interpretability report for (A) Escitalopram and (B) Duloxetine that provides the top 5 most salient variables for each treatment prediction. In this example, a question related to gastrointestinal symptoms is ranked first, which is interesting given that escitalopram is known to have superior tolerability, and the patient is noting significant gastrointestinal symptoms at baseline- a situation in which an SSRI with a more favorable tolerability profile would be preferred. In the second example, for duloxetine, some features are similar (e.g., gastrointestinal sensitivity) but is an interesting difference: hypochondriasis is included, which may be consistent with duloxetine’s known benefit for somatic symptoms.