Development and initial evaluation of a clinical prediction model for risk of treatment resistance in first-episode psychosis: Schizophrenia Prediction of Resistance to Treatment (SPIRIT)

Abstract

Background: A clinical tool to estimate the risk of treatment-resistant schizophrenia (TRS) in people with first-episode psychosis (FEP) would inform early detection of TRS and overcome the delay of up to 5 years in starting TRS medication.

Aims: To develop and evaluate a model that could predict the risk of TRS in routine clinical practice.

Method: We used data from two UK-based FEP cohorts (GAP and AESOP-10) to develop and internally validate a prognostic model that supports identification of patients at high-risk of TRS soon after FEP diagnosis. Using sociodemographic and clinical predictors, a model for predicting risk of TRS was developed based on penalised logistic regression, with missing data handled using multiple imputation. Internal validation was undertaken via bootstrapping, obtaining optimism-adjusted estimates of the model's performance. Interviews and focus groups with clinicians were conducted to establish clinically relevant risk thresholds and understand the acceptability and perceived utility of the model.

Results: We included seven factors in the prediction model that are predominantly assessed in clinical practice in patients with FEP. The model predicted treatment resistance among the 1081 patients with reasonable accuracy; the model's C-statistic was 0.727 (95% CI 0.723-0.732) prior to shrinkage and 0.687 after adjustment for optimism. Calibration was good (expected/observed ratio: 0.999; calibration-in-the-large: 0.000584) after adjustment for optimism.

Conclusions: We developed and internally validated a prediction model with reasonably good predictive metrics. Clinicians, patients and carers were involved in the development process. External validation of the tool is needed followed by co-design methodology to support implementation in early intervention services.

Keywords: First-episode schizophrenia; decision analysis; mixed methods; prognostic model; treatment resistant.

Fig. 1

Calibration plot for model 1, including all predictors (based on imputation data-set 31). E/O, expected/observed risk ratio; CITL, calibration-in-the-large; AUC, area under the curve.

Fig. 2

An example plot from the decision curve analysis (from imputation data-set 31). SPIRIT, schizophrenia prediction of resistance to treatment.

Fig. 3

(a) An example calibration plot for the Genetics and Psychosis (GAP) study (from imputation data-set 31). (b) An example calibration plot for the GAP study without schizophrenia diagnosis (from imputation data-set 31). E/O, expected/observed risk ratio; CITL, calibration-in-the-large; AUC, area under the curve.

Fig. 4

(a) An example calibration plot for the Aetiology and Ethnicity in Schizophrenia and Other Psychoses (AESOP) study (from imputation data-set 31). (b) An example calibration plot for the AESOP study without schizophrenia diagnosis (from imputation data-set 31). E/O, expected/observed risk ratio; CITL, calibration-in-the-large; AUC, area under the curve.