Simulation study to demonstrate biases created by diagnostic criteria of mental illnesses: major depressive episodes, dysthymia, and manic episodes

Abstract

Objectives: Composite diagnostic criteria alone are likely to create and introduce biases into diagnoses that subsequently have poor relationships with input symptoms. This study aims to understand the relationships between the diagnoses and the input symptoms, as well as the magnitudes of biases created by diagnostic criteria and introduced into the diagnoses of mental illnesses with large disease burdens (major depressive episodes, dysthymic disorder, and manic episodes).

Settings: General psychiatric care.

Participants: Without real-world data available to the public, 100 000 subjects were simulated and the input symptoms were assigned based on the assumed prevalence rates (0.05, 0.1, 0.3, 0.5 and 0.7) and correlations between symptoms (0, 0.1, 0.4, 0.7 and 0.9). The input symptoms were extracted from the diagnostic criteria. The diagnostic criteria were transformed into mathematical equations to demonstrate the sources of biases and convert the input symptoms into diagnoses.

Primary and secondary outcomes: The relationships between the input symptoms and diagnoses were interpreted using forward stepwise linear regressions. Biases due to data censoring or categorisation introduced into the intermediate variables, and the three diagnoses were measured.

Results: The prevalence rates of the diagnoses were lower than those of the input symptoms and proportional to the assumed prevalence rates and the correlations between the input symptoms. Certain input or bias variables consistently explained the diagnoses better than the others. Except for 0 correlations and 0.7 prevalence rates of the input symptoms for the diagnosis of dysthymic disorder, the input symptoms could not fully explain the diagnoses.

Conclusions: There are biases created due to composite diagnostic criteria and introduced into the diagnoses. The design of the diagnostic criteria determines the prevalence of the diagnoses and the relationships between the input symptoms, the diagnoses, and the biases. The importance of the input symptoms has been distorted largely by the diagnostic criteria.

Keywords: bias; forward-stepwise regression; frailty; index mining; the health and retirement study.

Figure 1

The prevalence rates of an intermediate variable for the diagnosis of major depressive episodes by assumed input symptom prevalence and correlations. The intermediate variable is ‘significant unintentional weight loss or gain’ and the input symptoms are ‘significant unintentional weight loss’ and ‘significant unintentional weight gain.’ The black line represents the situation where the prevalence rates of the input symptoms are the same as that of the intermediate variable. Lines above the black lines have the prevalence rates of the intermediate variable larger than those of the input symptoms. CI, confidence interval.

Figure 2

The prevalence rates of dysthymic disorder by assumed input symptom prevalence and correlations. Dysthymic disorder is diagnosed when both the major (depressed mood most of the day for more days than not, for at least 2 years) and minor criteria (at least two of the six items) are met. The black line represents the situation where the prevalence rates of the input symptoms are the same as those of dysthymic disorder. Lines below the black lines have dysthymic disorder prevalence rates lower than those of the input symptoms. CI, confidence interval.

Figure 3

The prevalence rates of major depressive episodes by assumed input symptom prevalence and correlations. Major depressive episodes are diagnosed when both major and minor criteria are confirmed. The black line represents the situation where the prevalence rates of the input symptoms are the same as that of major depressive episodes. Lines below the black lines have the prevalence rates of major depressive disorder lower than those of the input symptoms. CI, confidence interval.

Figure 4

The prevalence rates of manic episodes by assumed input symptom prevalence and correlations. Manic episodes are diagnosed when the symptoms present as described in the diagnostic manual. The black line represents the situation where the prevalence rates of manic episodes are the same as those of the input symptoms. Lines below the black lines have prevalence rates of manic episodes lower than those of the input symptoms. CI, confidence interval.

Figure 5

The approximation of the diagnosis of dysthymic disorder by input symptoms, bias variables and both, measured by R-squared. The diagnosis of dysthymic disorder is approximated by all variable, including input symptoms and bias variables, using forward-stepwise regression. The selection of the variables was determined by adjusted R-squared. Circles are the maximal adjusted R-squared achieved by the regression with input symptoms, bias variables, or both of them. See table 4 for the details in the input symptoms and the bias variables. The assumed correlations between the input symptoms are 0.4 and the assumed prevalence rates of the input symptoms are 0.7 in this figure.