Digital phenotyping of negative symptoms: the relationship to clinician ratings

Abstract

Negative symptoms are a critical, but poorly understood, aspect of schizophrenia. Measurement of negative symptoms primarily relies on clinician ratings, an endeavor with established reliability and validity. There have been increasing attempts to digitally phenotype negative symptoms using objective biobehavioral technologies, eg, using computerized analysis of vocal, speech, facial, hand and other behaviors. Surprisingly, biobehavioral technologies and clinician ratings are only modestly inter-related, and findings from individual studies often do not replicate or are counterintuitive. In this article, we document and evaluate this lack of convergence in 4 case studies, in an archival dataset of 877 audio/video samples, and in the extant literature. We then explain this divergence in terms of "resolution"-a critical psychometric property in biomedical, engineering, and computational sciences defined as precision in distinguishing various aspects of a signal. We demonstrate how convergence between clinical ratings and biobehavioral data can be achieved by scaling data across various resolutions. Clinical ratings reflect an indispensable tool that integrates considerable information into actionable, yet "low resolution" ordinal ratings. This allows viewing of the "forest" of negative symptoms. Unfortunately, their resolution cannot be scaled or decomposed with sufficient precision to isolate the time, setting, and nature of negative symptoms for many purposes (ie, to see the "trees"). Biobehavioral measures afford precision for understanding when, where, and why negative symptoms emerge, though much work is needed to validate them. Digital phenotyping of negative symptoms can provide unprecedented opportunities for tracking, understanding, and treating them, but requires consideration of resolution.

Keywords: biobehavioral; computational; deficit; digital phenotyping; negative; psychiatry; schizophrenia.

Fig. 1.

Clinical rating and biobehavioral-based measures of negative symptoms from 4 archived clinical interviews conducted by either Dr. Brian Kirkpatrick or Dr. Gregory Strauss. For NP1 and NP2, shaded cells are inconsistent with clinical ratings (ie, less severe negative symptom-behavior compared to the interviewers and non-negative patients). var = variability, sec = seconds, AL = conceptually taps alogia, BA = conceptually taps blunted vocal affect, FA = conceptually taps blunted facial affect, OTHER = potentially taps anhedonia or other aspect of negative symptoms, C = analyzed using the Computerized Assessment of Natural Speech, L = analyzed using the Linguistic Inquiry and Word Count system, F = analyzed using Facereader software. 1 = standard deviation of fundamental frequency (converted to semitones), computed within each utterance and averaged across all utterances, 2 = standard deviation of intensity (in dB), computed within each utterance and averaged across all utterances, 3 = based on first formant, in semitones, 4 = based on second formant, in semitones, 5 = percentage of total words, based on match to predefined dictionary, 6 = analysis of face from each video frame by matching to an “ideal” neutral face algorith, based on .00 (no fit) to 1.00 (ideal fit), averaged across all frames, ⁷Video data unavailable for the interviewers.

Fig. 2.

Relative informational value of computerized facial analysis versus clinical ratings for participant NP2 and P1. Computerized facial data are scaled as a function of time (over the span of the clinical interview) and spectral component (ie, facial affect type).