Reporting Quality of Randomized Controlled Trials of Periodontal Diseases in Journal Abstracts-A Cross-sectional Survey and Bibliometric Analysis

Abstract

Objective: Randomized controlled trials (RCTs) by proper design, conduct, analysis, and reporting provide reliable information in clinical care. Reporting of RCT abstracts is of equal importance as there is evidence that many clinicians will change their clinical decisions based on RCT abstracts. The reporting quality of RCT abstracts has been suboptimal. It is not clear whether the reporting quality is related to the journal metrics. The main objective of this study is to conduct a cross-sectional survey to evaluate the reporting quality of RCTs of periodontal diseases in journal abstracts and to perform a bibliometric analysis. The null hypothesis was that there is no association between the journal metrics (5-year impact factor, Eigenfactor score, and Article Influence Score), abstract metrics (word count, and number of authors), journal endorsement of Consolidated Standards of Reporting Trials (CONSORT), and the overall quality of reporting of CONSORT RCT abstract-modified checklist questions.

Materials: CONSORT RCT abstract extension checklist with explanation and elaboration was used and modified to assess the quality of reporting of RCT abstracts of periodontal diseases in the journal abstracts in the year 2012. Bibliometric analysis of journal metrics (5-year impact factor, Eigenfactor score, and Article Influence Score) and abstract metrics (number of authors and abstract word count), the geographic distribution, and the CONSORT-endorsing journal abstracts was compared with the reporting quality of RCT abstracts in periodontal diseases. Calibration and intrarater agreement were done before the data collection and analysis. A second reviewer was consulted for independent evaluation and clarification as needed. For descriptive analysis, the values of continuous variables were expressed as median and interquartile ranges (IQRs) and as proportion percent for binary categorical variables. For association analysis between the binary (yes/no) response variable and the continuous variable, the Mann-Whitney test (for independent samples) was used. For examining the association between 2 categorical variables, Fisher's exact test was used. The chi-square test was performed to examine the association between 2 sets of binary response variables (yes/no). A P value of < .05 was considered statistically significant. All analyses were conducted using SAS, version 9.4.

Results: A total of 198 RCT abstracts of periodontal diseases in the year 2012 from 57 journals were included in the study. Fifteen journals, listed as endorsers of CONSORT, contributed 108 RCT abstracts. Four journals (Journal of Periodontology, Journal of Clinical Periodontology, Clinical Oral Implants Research, and European Journal of Oral Implantology) contributed 84 of 198 RCT abstracts in 2012. European countries contributed the majority (n = 81, 40.91%) of RCT abstracts. Among 31 countries in this study, United States contributed the most RCTs (n = 28, 14.14%) followed by India (24, 12.12%), Italy (n = 22, 11.11%), and Brazil (n = 20, 10.1%). The frequency of journal metrics were 5-year impact factor (median 2.316; IQR: 1.439-2.970); Eigenfactor score (0.00474; 0.00202-0.01395); and Article Influence Score (0.553; 0.382-0.755). The number of authors in 198 RCT abstracts ranged between 2 and 20 (median n = 5, IQR: 4-6), whereas the word count ranged between 48 and 569 (median 235, IQR: 205-269). All RCT abstracts reported the experimental interventions (checklist question #5, frequency 100%). Some items were almost always reported-participant eligibility criteria (#3, 99%); comparison interventions (#6, 99.5%); specific objective or hypothesis (#7, 99.5%); primary outcome (#8, 99.5%); and reporting trial results as a summary (#16, 98.5%). All RCT abstracts never reported how the allocations were concealed (#11, 0) and the source of funding for the trials (#23, 0). Some items were almost always never reported-the number of participants included in the analysis for each intervention (#15, 2%); trial registration number (#21, 2.5%); name of trial register (#22, 2.5%); and how the randomization or sequence generation was done (#22). Dismal reporting was noted in many checklist questions including the identification of the study as randomized in the title #1, 51%; design of the trial #2, 32.8%; trial setting #4, 3.5%; randomization #10, 3.5%; blinding #12, 21.7%; details about blinding #13, 8.1%; number of participants randomized to each intervention #14, 26.3%; effect size #17, 13.6%; precision of the estimate of the effect #18, 6.1%; and adverse effects #19, 14.1%. Strikingly, there was a very high reporting of statistical significance #25, 92.4%. European countries, in particular, reported relatively better than other countries in essential questions such as #17 effect size reporting, and #18 precision (uncertainty), which have been largely unreported by rest of the countries. Finally, despite the majority of RCTs published in 2012 were by CONSORT-endorsing journals, there was no difference in the quality of reporting in majority of checklist items when compared with journals not listed as CONSORT endorsers. With few exceptions, there was no statistically significant association between the majority of the CONSORT RCT abstract checklist questions and the journal metrics and abstract metrics analyzed in this study. Unexpectedly, lower ranking journals in journal metrics reported certain essential checklist questions relatively better.

Conclusion: The reporting quality of RCT of periodontal diseases in the journal abstracts published in 2012 needs substantial improvement. These items have been laid out in this study to help all stakeholders-authors, clinicians, researchers, peer reviewers, journal editors, and publishers to take note and help with the improvement of the same. Despite few significant associations in the bibliometric factors analyzed with better reporting, the results overall led to the failure to reject the null hypothesis that there is no association between the journal metrics, word count, and number of authors and the quality of reporting of CONSORT RCT abstract-modified checklist questions.

Keywords: Abstracts; Bibliometrics; Consolidated Standards of Reporting Trials (CONSORT); Journal Impact Factor; Periodontal diseases; Randomized Controlled Trials.