An appraisal of the utility or futility of ENT consultant postal questionnaires

Abstract

Despite an increase in ENT postal questionnaires, the quality of their methodology has been questioned (Ramphul et al. in J Laryngol Otol 119:175-178, 1). This retrospective study examined whether quality and utility of such questionnaires published since 2005 has improved. Seventeen consultant postal questionnaires published between 2005 and 2012 were reviewed. Quality of questionnaires was assessed using a 30-point score based on compliance with 15 criteria previously established to evaluate postal questionnaire study-design (Ramphul et al. in J Laryngol Otol 119:175-178, 1). Citation rates were used as an indicator of utility. The specific comments made in each citing paper was reviewed providing information on whether questionnaire findings (a) had an impact on clinical practice, (b) were the citing comments positive, (c) negative or (d) non-specific. Recurrent methodological flaws were identified in all questionnaires. The average score assigned was 44 %, versus 32 % previously reported (Ramphul et al. in J Laryngol Otol 119:175-178, 1) (P < 0.01, Student's t test). The low citation rate demonstrates poor utility for postal questionnaires. Citations were general non-specific referencing with no clear indication that questionnaire findings positively impacted clinical practice. In conclusion, although the quality of ENT postal questionnaire has improved since the original study (Ramphul et al. in J Laryngol Otol 119:175-178, 1), important recurring methodological flaws still exist. The poor utility, based on low citation rates, also reflects the continued deficiencies in design quality. It is recommended that authors of questionnaire-based research should ensure that guidelines for questionnaire design are adhered in order to improve the validity of findings and hence impact on clinical practice.