Development and Validation of Predictive Model for a Diagnosis of First Episode Psychosis Using the Multinational EU-GEI Case-control Study and Modern Statistical Learning Methods

Abstract

Background and hypothesis: It is argued that availability of diagnostic models will facilitate a more rapid identification of individuals who are at a higher risk of first episode psychosis (FEP). Therefore, we developed, evaluated, and validated a diagnostic risk estimation model to classify individual with FEP and controls across six countries.

Study design: We used data from a large multi-center study encompassing 2627 phenotypically well-defined participants (aged 18-64 years) recruited from six countries spanning 17 research sites, as part of the European Network of National Schizophrenia Networks Studying Gene-Environment Interactions study. To build the diagnostic model and identify which of important factors for estimating an individual risk of FEP, we applied a binary logistic model with regularization by the least absolute shrinkage and selection operator. The model was validated employing the internal-external cross-validation approach. The model performance was assessed with the area under the receiver operating characteristic curve (AUROC), calibration, sensitivity, and specificity.

Study results: Having included preselected 22 predictor variables, the model was able to discriminate adults with FEP and controls with high accuracy across all six countries (ranges_AUROC = 0.84-0.86). Specificity (range = 73.9-78.0%) and sensitivity (range = 75.6-79.3%) were equally good, cumulatively indicating an excellent model accuracy; though, calibration slope for the diagnostic model showed a presence of some overfitting when applied specifically to participants from France, the UK, and The Netherlands.

Conclusions: The new FEP model achieved a good discrimination and good calibration across six countries with different ethnic contributions supporting its robustness and good generalizability.

Keywords: cannabis use; diagnostic prediction modeling/risk prediction; psychosis/diagnostic factors.

Fig. 1.

Internally–externally validated calibration plots for the prediction model of FEP in the United Kingdom, the Netherlands, France, Spain, Italy, and Brazil. Squares illustrate risk groups by fourths of equally spaced model-estimated risks, through which the linear line was fitted (red). The smoothed loess curve (orange) was fitted based on individual data points. The 45° line (gray) represents perfect calibration where model-estimated equal actual risk. The histogram on the upper margin represents the distribution of model-estimated risks.