Comparison of analytical methods for cluster randomised trials: an example from a primary care setting

Abstract

Background: Cluster randomisation is commonly used to evaluate educational and organisational interventions in primary care. We conducted a study where 66 general practices in the North East of Scotland were randomised to receive guidelines and access to a fast-track investigation service for two common urological conditions. Patients were identified from referral letters and recruited upon referral to secondary care. Although these urological conditions are common in secondary care, the number of referrals per general practice can be small; this created a number of issues for the analysis.

Methods: Three general approaches in the analysis of cluster randomised trials; the adjustment of standard tests; analysis at cluster level; and advanced statistical techniques (random effects models and generalised estimating equations) were applied to data from the above trial. The effect of the intervention on both a continuous and a dichotomous outcome was investigated.

Results: Spuriously low P values were obtained when conventional tests (which do not account for clustering in the data) were applied. Cluster level analysis of the dichotomous outcome with no account for cluster size resulted in a different conclusion compared with cluster level analysis with weighting, standard tests with adjustment and advanced statistical methods.

Discussion: Cluster randomised trials are becoming increasingly common in primary care. Where recruitment of individual patients is generated by referral from primary to secondary care it is likely that the trial will suffer from inherent weaknesses: not all clusters randomised contribute to the analysis; there is the likelihood of single size clusters and variable cluster sizes. Our analysis indicated that the different approaches produced consistent results across continuous outcomes, but for dichotomous outcomes in the cluster level analysis, failure to weight observations would have resulted in a different conclusion.