Development and content validity of the Person Experiences Interview Survey (PEIS): a measure of the mental health services experiences of people with developmental disabilities

Abstract

Purpose: People with developmental disabilities and mental health service experiences have a right to be included in healthcare decisions, including the evaluation of their mental health services and providers. However, few self-report measures address this need. This study aimed to fill this gap by developing and evaluating the content validity, including comprehension, relevance, and comprehensiveness, of the Person Experiences Interview Survey (PEIS) with people with developmental disabilities and mental health experiences.

Methods: The research team established a measurement framework based on the Family Experiences Interview Survey (FEIS), resulting in 21 PEIS items that were written in collaboration with young adults with developmental disabilities and mental health service experiences. Comprehension, relevance, and comprehensiveness were evaluated through cognitive interviews with people with developmental disabilities and mental health service experiences (respondents; n = 9) ages 23-49 years. Comprehensiveness and relevance were also evaluated in focus groups with family caregivers (n = 9) and mental health providers (n = 10) who serve this population. Two researchers independently coded open-ended responses to the PEIS for comprehension. A content validity index (CVI), indicating relevance, was calculated for each participant group for each item, and comprehensiveness was rated for item sets.

Results: Fifteen of the 21 items met the criteria of ≥80% comprehension, with 89-100% of responses containing all or some intended information. All items met the CVI ≥80% criterion in at least two of the three groups. In all item sets, between 1 and 4 family members or providers felt one question was missing. Respondents used the response scale in a manner that corresponded with their open-ended descriptions, and family caregivers and providers had positive feedback about the response scale's visual cues and number of choices. Using these findings, four items were removed and six items were revised, resulting in a 17-item measure.

Conclusion: This study presents a novel and promising measure, the Person Experiences Interview Survey (PEIS). It also demonstrates that the employment of accessible methods allows people with developmental disabilities to meaningfully evaluate mental health services and providers. The PEIS shows great promise for application in the field by engaging those directly involved in the evaluation of mental health services and providers.

Keywords: developmental disabilities (MESH term); healthcare surveys (MESH term); mental health services (MESH term); patient-reported experience measure; self-report.

Figure 1

PEIS response scale with visual supports.