From abstract to impact in cardiovascular research: factors predicting publication and citation

Abstract

Aims: Through a 4-year follow-up of the abstracts submitted to the European Society of Cardiology Congress in 2006, we aimed at identifying factors predicting high-quality research, appraising the quality of the peer review and editorial processes, and thereby revealing potential ways to improve future research, peer review, and editorial work. METHODS AND RESULTS All abstracts submitted in 2006 were assessed for acceptance, presentation format, and average reviewer rating. Accepted and rejected studies were followed for 4 years. Multivariate regression analyses of a representative selection of 10% of all abstracts (n= 1002) were performed to identify factors predicting acceptance, subsequent publication, and citation. A total of 10 020 abstracts were submitted, 3104 (31%) were accepted for poster, and 701 (7%) for oral presentation. At Congress level, basic research, a patient number ≥ 100, and prospective study design were identified as independent predictors of acceptance. These factors differed from those predicting full-text publication, which included academic affiliation. The single parameter predicting frequent citation was study design with randomized controlled trials reaching the highest citation rates. The publication rate of accepted studies was 38%, whereas only 24% of rejected studies were published. Among published studies, those accepted at the Congress received higher citation rates than rejected ones.

Conclusions: Research of high quality was determined by study design and largely identified at Congress level through blinded peer review. The scientometric follow-up revealed a marked disparity between predictors of full-text publication and those predicting citation or acceptance at the Congress.

Figure 1

Fate of studies submitted to the ESC Congress 2006. Overview of submission, acceptance at Congress level, and follow-up for full-text publication, and citation within 2 years following publication. All abstracts were characterized according to a set of pre-specified parameters. Association of these factors with the acceptance at Congress level, with full-text publication, and subsequent citation was assessed by univariable and mutlivariable regression analyses. Percentages refer to the respective preceding level (n). (n* refers to the representative random selection of 10% (1002) of all abstracts submitted (10 020), Ø published, not published; IF, impact factor).

Figure 2

Origin of research submitted to the ESC Congress 2006. A total of 10 020 abstracts were submitted to the scientific sessions of the ESC 2006. Based on the assessment of a representative subsample of 10% (n= 1002) submitted studies originated from 63 different countries of all five continents. About one-third of all studies were submitted from the American continent, 43% from Western Europe, and 14% from Eastern Europe and Russia. Less than 10% of studies originated from Africa and Asia/Oceania.

Figure 3

Peer review and editorial process of the ESC Congress 2006. All abstracts submitted were peer-reviewed in a blinded fashion by three to eight expert reviewers, and graded on a scale from 1 to 10. (A) Reviewer ratings and editorial decisions. Accepted studies had received significantly higher ratings than rejected studies; studies for oral presentation had been ranked significantly higher than those for poster presentations. The cut-off for acceptance at 6.5 was calculated using the Yuden's index (sensitivity 0.97, specificity 0.92). (B) Receiver operating characteristic (ROC) curve analyses after the scientometric follow-up of a representative subsample of 10% of all abstracts submitted (n= 1002) revealed a predictive value of average reviewer ratings of 69.4%. (C) Spearman's correlation of average reviewer ratings with acceptance at the Congress, subsequent full-text-publication and citation rates uncovered a significant positive correlation in all three cases. (D) Comparison of the numbers of 2-year citations of both accepted and rejected, subsequently published studies. Analyses revealed that rejected and subsequently published studies were cited significantly less frequently compared with accepted and subsequently published studies; citation frequencies between different presentation formats did not differ. AUC, area under the curve, significance level (P< 0.017) adjusted for multiple comparisons using Bonferroni's correction, interquartile ranges, whiskers indicate minima and maxima.

Figure 4

Key findings: predictors of acceptance at Congress level, full-text publication, and frequent citation, and their validation with regard to scientific impact. We identified parameters predicting acceptance, subsequent full-text publication, and high citation rates (≥10 two-year citations). Parameters predicting scientific impact (+), no association with scientific impact (−). All findings, odds ratios, and confidence intervals (CI) refer to multivariable regression analyses based on a random 10% selection of all abstracts submitted to the ESC Congress 2006 (n= 1002), margin of error < 0.01.