Citation indexes do not reflect methodological quality in lung cancer randomised trials

Abstract

Background: Citation factors are applied to assess scientific work despite the fact that they were developed commercially in order to compare competing journals. The aim of the present study was to determine whether there is a relationship between citation factors and a trial's methodological quality using published randomised trials in lung cancer clinical research. Material and methods All of the randomised trials included in nine systematic reviews performed by the European Lung Cancer Working Party (ELCWP) were assessed using two quality scales (Chalmers and ELCWP).

Results: One hundred and eighty-one articles were eligible. The median overall ELCWP and Chalmers quality scores were 61.8% and 49.0%, respectively, with a correlation coefficient (r(s)) of 0.74 (P <0.001). A weak association was observed between citation factors and quality scores with the respective correlation coefficients ranging from 0.18 to 0.40 (ELCWP scale) and from 0.21 to 0.38 (Chalmers scale). American authors published trials significantly more often in journals with high citation factors than European or non-American authors (P <0.0001), despite no better methodological quality. Positive trials, which were significantly more likely to be published in journals with higher citation factors, were of no better quality than negative ones.

Conclusion: Journals with higher citation factors do not appear to publish clinical trials with higher levels of methodological quality, at least for trials in the field of lung cancer research.