The analysis of 168 randomised controlled trials to test data integrity

Abstract

The purpose of this study was to use some statistical methods to assess if randomised controlled trials (RCTs) published by one particular author (Fujii) contained data of unusual consistency. I searched seven electronic databases, retrieving 168 RCTs published by this author between 1991 and July 2011. I extracted rates for categorical variables and means (SDs) for continuous variables, and compared these published distributions with distributions that would be expected by chance. The published distributions of 28/33 variables (85%) were inconsistent with the expected distributions, such that the likelihood of their occurring ranged from 1 in 25 to less than 1 in 1 000 000 000 000 000 000 000 000 000 000 000 (1 in 10(33)), equivalent to p values of 0.04 to < 1 × 10(-33) , respectively. In 141 human studies, 13/13 published continuous variable distributions were inconsistent with expected, their likelihoods being: weight < 1 in 10(33) ; age < 1 in 10(33) ; height < 1 in 10(33) ; last menstrual period 1 in 4.5 × 10(15) ; baseline blood pressure 1 in 4.2 × 10(5) ; gestational age 1 in 28; operation time < 1 in 10(33) ; anaesthetic time < 1 in 10(33) ; fentanyl dose 1 in 6.3 × 10(8) ; operative blood loss 1 in 5.6 × 10(9) ; propofol dose 1 in 7.7 × 10(7) ; paracetamol dose 1 in 4.4 × 10(2) ; uterus extrusion time 1 in 33. The published distributions of 7/11 categorical variables in these 141 studies were inconsistent with the expected, their likelihoods being: previous postoperative nausea and vomiting 1 in 2.5 × 10(6) ; motion sickness 1 in 1.0 × 10(4) ; male or female 1 in 140; antihypertensive drug 1 in 25; postoperative headache 1 in 7.1 × 10(10) ; postoperative dizziness 1 in 1.6 × 10(6) ; postoperative drowsiness 1 in 3.8 × 10(4) . Distributions for individual RCTs were inconsistent with the expected in 96/134 human studies by Fujii et al. that reported more than two continuous variables, their likelihood ranging from 1 in 22 to 1 in 140 000 000 000 (1 in 1.4 × 10(11)), compared with 12/139 RCTs by other authors. In 26 canine studies, the distributions of 8/9 continuous variables were inconsistent with the expected, their likelihoods being: right atrial pressure < 1 in 10(33) ; diaphragmatic stimulation (100 Hz) < 1 in 10(33) ; pulmonary artery occlusion pressure < 1 in 10(33) ; diaphragmatic stimulation (20 Hz) < 1 in 10(33) ; heart rate 1 in 6.3 × 10(10) ; mean pulmonary artery pressure 1 in 2.2 × 10(14) ; mean arterial pressure 1 in 6.3 × 10(7) ; cardiac output 1 in 110. Distributions were inconsistent with the expected in 21/24 individual canine studies that reported more than two continuous variables, their likelihood ranging from 1 in 345 to 1 in 51 000 000 000 000 (1 in 5.1 × 10(13)).