Assessment of Population Well-being With the Mental Health Quotient: Validation Study

Abstract

Background: The Mental Health Quotient (MHQ) is an anonymous web-based assessment of mental health and well-being that comprehensively covers symptoms across 10 major psychiatric disorders, as well as positive elements of mental function. It uses a novel life impact scale and provides a score to the individual that places them on a spectrum from Distressed to Thriving along with a personal report that offers self-care recommendations. Since April 2020, the MHQ has been freely deployed as part of the Mental Health Million Project.

Objective: This paper demonstrates the reliability and validity of the MHQ, including the construct validity of the life impact scale, sample and test-retest reliability of the assessment, and criterion validation of the MHQ with respect to clinical burden and productivity loss.

Methods: Data were taken from the Mental Health Million open-access database (N=179,238) and included responses from English-speaking adults (aged≥18 years) from the United States, Canada, the United Kingdom, Ireland, Australia, New Zealand, South Africa, Singapore, India, and Nigeria collected during 2021. To assess sample reliability, random demographically matched samples (each 11,033/179,238, 6.16%) were compared within the same 6-month period. Test-retest reliability was determined using the subset of individuals who had taken the assessment twice ≥3 days apart (1907/179,238, 1.06%). To assess the construct validity of the life impact scale, additional questions were asked about the frequency and severity of an example symptom (feelings of sadness, distress, or hopelessness; 4247/179,238, 2.37%). To assess criterion validity, elements rated as having a highly negative life impact by a respondent (equivalent to experiencing the symptom ≥5 days a week) were mapped to clinical diagnostic criteria to calculate the clinical burden (174,618/179,238, 97.42%). In addition, MHQ scores were compared with the number of workdays missed or with reduced productivity in the past month (7625/179,238, 4.25%).

Results: Distinct samples collected during the same period had indistinguishable MHQ distributions and MHQ scores were correlated with r=0.84 between retakes within an 8- to 120-day period. Life impact ratings were correlated with frequency and severity of symptoms, with a clear linear relationship (R²>0.99). Furthermore, the aggregate MHQ scores were systematically related to both clinical burden and productivity. At one end of the scale, 89.08% (8986/10,087) of those in the Distressed category mapped to one or more disorders and had an average productivity loss of 15.2 (SD 11.2; SEM [standard error of measurement] 0.5) days per month. In contrast, at the other end of the scale, 0% (1/24,365) of those in the Thriving category mapped to any of the 10 disorders and had an average productivity loss of 1.3 (SD 3.6; SEM 0.1) days per month.

Conclusions: The MHQ is a valid and reliable assessment of mental health and well-being when delivered anonymously on the web.

Keywords: anxiety; attention deficit disorder with hyperactivity; autistic disorder; behavioral symptoms; depression; diagnosis; global health; internet; mHealth; mental disorders; mental health; methods; population health; psychiatry; psychopathology; public health; social determinants of health; symptom assessment.

Figure 1

Reliability of the Mental Health Quotient (MHQ). (A) The average ratings of each of the 27 MHQ spectrum elements (left) and 20 MHQ problem elements (right) in 4 separate samples of the MHQ obtained over a similar period were indistinguishable (each bar is a sample). (B) Distribution of ratings for an example MHQ element (Self-image) in each of the 4 samples (each line is a sample). (C) Distribution of MHQ scores in each of the 4 samples (each line is a sample) for an example MHQ element (Self-image).

Figure 2

Test-retest reliability of the Mental Health Quotient (MHQ). MHQ score for the assessment retake (retest) versus MHQ score for the first take (test). The red line represents y=x. MHQ: Mental Health Quotient.

Figure 3

Example relationship between life impact and frequency and severity. (A) The selection on the Mental Health Quotient (MHQ) life impact scale for Feelings of sadness, distress, and hopelessness was linearly related to the frequency of these feelings. (B) The selection on the MHQ life impact scale was similarly linearly related to the severity of the symptom. (C) Frequency × severity was nonlinearly related to the MHQ life impact selection.

Figure 4

Relationship between Mental Health Quotient (MHQ) score and clinical symptoms and diagnosis. (A) The average number of symptoms (life impact ≥8 for problem elements and ≤1 for spectrum elements) decreased as the MHQ score increased. The grey area represents the negative side of the scale or the MHQ score categories of Distressed and Struggling (all panels). (B) Approximately 97.83% (3926/4013) of those with the lowest MHQ scores (−100 to −75) mapped to at least one of 10 major clinical disorders and decreased systematically. (C) The average number of disorders per person decreased with MHQ score.

Figure 5

Relationship between Mental Health Quotient (MHQ) score and productivity. (A) The days unable to work in the past month decreased nonlinearly as the MHQ score increased (closed circles, exponential fit, R²=0.98). Employed people (open circles) with low MHQ scores missed fewer days of work or productive activity. (B) The days in the past month with reduced productivity (presenteeism) decreased linearly as the MHQ score increased (closed circles, exponential fit, R²=0.98). Employed people (open circles) with low MHQ scores had more days of presenteeism. (C) Total productivity loss for employed (dotted line) and all respondents together (solid line) as a function of the MHQ score (calculated as days missed + n * days with reduced productivity, where n is assumed to be a range between 0.2 [lower dotted or solid line] and 0.5 [upper dotted or solid line]).