External validation and extension of the NAPLS-2 and SIPS-RC personalized risk calculators in an independent clinical high-risk sample

Abstract

Early identification of individuals likely to develop psychosis is a priority for the field, resulting in the development of risk calculators that provide personalized estimates that an individual at clinical high-risk (CHR) will develop psychosis. The North American Prodrome Longitudinal Study (NAPLS) consortium and Shanghai At-Risk for Psychosis program have recently developed such calculators (NAPLS-2/SIPS-RC, respectively), but their discrimination performance has never been examined within the same sample. Moreover, validation studies of NAPLS-2 are limited in number and the SIPS-RC has not been cross-validated in a North American sample. The present research (N = 68) used the area under the receiver operating characteristic curve (AUC) to examine the accuracy of the NAPLS-2 and SIPS-RC calculators for discriminating CHR converters and non-converters, as well as extend their use by examining their ability to predict illness progression over a two-year period. For conversion, the NAPLS-2 and SIPS-RC risk calculators demonstrated moderate (AUC = 0.71) and fair (AUC = 0.65) discrimination performance, respectively. Both calculators provided moderate accuracy for discriminating illness progression over two-years (NAPLS-2 AUC = 0.71/ SIPS-RC AUC = 0.76). We discuss implications for researchers and practitioners interested in using the NAPLS-2 and/or SIPS-RC and identify important steps for future research.

Keywords: High-risk syndromes; Prodrome; Psychosis; Psychosis-risk prediction; Psychotic disorders; Risk calculator; Risk models.

Figure 1.

Panel A: Receiver operator characteristics curves (ROC) for the NAPLS-2 and SIPS-RC risk calculators discriminating subsequent conversion. Panel B: ROC curves for the NAPLS-2 and SIPS-RC risk calculator discriminating CHR youth with and without illness progression over a one-year period. Panel C: ROC curves for the NAPLS-2 and SIPS-RC risk calculators discriminating CHR youth with and without illness progression over a two-year period. Note: AUC = area under the curve; SE = standard error; CI = confidence interval.