Assessing the validity of a Parkinson's care evaluation: the PRIME-NL study

Abstract

Introduction: The PRIME-NL study prospectively evaluates a new integrated and personalized care model for people with parkinsonism, including Parkinson's disease, in a selected region (PRIME) in the Netherlands. We address the generalizability and sources of selection and confounding bias of the PRIME-NL study by examining baseline and 1-year compliance data.

Methods: First, we assessed regional baseline differences between the PRIME and the usual care (UC) region using healthcare claims data of almost all people with Parkinson's disease in the Netherlands (the source population). Second, we compared our questionnaire sample to the source population to determine generalizability. Third, we investigated sources of bias by comparing the PRIME and UC questionnaire sample on baseline characteristics and 1-year compliance.

Results: Baseline characteristics were similar in the PRIME (n = 1430) and UC (n = 26,250) source populations. The combined questionnaire sample (n = 920) was somewhat younger and had a slightly longer disease duration than the combined source population. Compared to the questionnaire sample in the PRIME region, the UC questionnaire sample was slightly younger, had better cognition, had a longer disease duration, had a higher educational attainment and consumed more alcohol. 1-year compliance of the questionnaire sample was higher in the UC region (96%) than in the PRIME region (92%).

Conclusion: The generalizability of the PRIME-NL study seems to be good, yet we found evidence of some selection bias. This selection bias necessitates the use of advanced statistical methods for the final evaluation of PRIME-NL, such as inverse probability weighting or propensity score matching. The PRIME-NL study provides a unique window into the validity of a large-scale care evaluation for people with a chronic disease, in this case parkinsonism.

Keywords: Confounding bias; Epidemiology; Generalizability; Healthcare evaluation; Healthcare model; Parkinsonism; Parkinson’s disease; Selection bias; Validity.

Fig. 1

Directed Acyclic Graph (DAG) of possible potential sources of bias that may influence the eventual evaluation of the PRIME-NL study. In orange, a highly simplified version of the effect of PRIME care is displayed (middle pathway): improved care can ameliorate motor and non-motor symptoms which in turn reduce the amount of complications and improve the quality of life. However, for an adequate evaluation of the PRIME Parkinson care model, several methodological challenges and potential sources of bias need to be identified. First, PRIME Parkinson care has been implemented in a specific, non-randomized region of the Netherlands which might be different from the rest of the Netherlands (UC region) at baseline. The regions can differ in sociodemographic factors that impact the presence of symptoms, complications, and quality of life (top pathway). Sociodemographic factors can thereby introduce confounding bias, e.g., the PRIME participants are older, and older age is associated with more symptoms and more complications, making PRIME look worse on the final evaluation when not correcting for age. Second, we might have differentially recruited people from the source populations into the questionnaire sample, e.g., through the letter by the neurologists in the PRIME region. This letter might have reached specific subgroups of participants in the PRIME region, e.g., people with more symptoms, introducing selection bias. Third, collider bias might create an artificial association between the region and outcomes when differential loss to follow-up occurs. For example, if we assume that we have recruited more affected people in the PRIME region and participants with more symptoms are less likely to return their questionnaire, the PRIME region will appear worse compared to UC in which fewer highly affected participants are retained (bottom pathway). We have not illustrated information bias in this DAG since participants were unaware of the study group at baseline. However, at follow-up, because the study is unblinded, they will be aware and this could introduce differential measurement error

Fig. 2

Overview of the comparisons made. First, we assessed regional baseline differences between the PRIME and the usual care (UC) region using healthcare claims data of almost all people with Parkinson’s disease in the Netherlands (the source population) (1). Second, we compared the combined questionnaire sample of participants from both regions to the source population to determine if the questionnaire sample findings will be generalizable to all people with PD in the Netherlands (2). Third, to assess selection and confounding bias between the two regions, we compared the PRIME and UC questionnaire sample on baseline characteristics (3a) and investigated whether there is differential 1-year compliance (3b and 3c)

Fig. 3

Illustration of the loss to follow-up during the first year of the PRIME-NL study