An experimental study of the influence of individual participant characteristics on formal consensus development

Abstract

Objectives: The aim of this study was to examine the influence of participants' characteristics on the results produced by formal consensus methods.

Methods: The approach was an experimental study of 346 participants in 20 groups rating the appropriateness of four mental health interventions for the treatment of chronic fatigue syndrome, irritable bowel syndrome, and chronic back pain. There were four factors in the design: systematic literature review provided or not, decisions made under realistic or "ideal" resource assumptions, clinically mixed (general practitioners and mental health professionals) or homogenous group (general practitioners only), convened or mail-only group. A group's rating was defined as the median of participants' ratings. The influence of participants' characteristics (age, sex, and specialty) was examined using multilevel models.

Results: The largest differences were between the GPs and mental health professionals, both in their initial ratings of the different interventions, and in how much they altered their ratings between rounds. There were smaller but statistically significant (p<.05) differences between specialty and age groups in initial ratings for the treatment (by whatever means) of different conditions, and for certain conditions women increased their ratings more than men. Women rated intervention more favorably when assuming "ideal" rather than realistic levels of resources, but men did not.

Conclusions: Our findings support the practice of treating professional specialty as an important determinant of the results in consensus panels.