False Alarms, Pseudoepidemics, and Reality: A Case Study with American College of Obstetricians and Gynecologists Practice Bulletins

Abstract

Objectives: The objectives of this review are to (1) ascertain the frequency with which odd ratio (OR) and relative risk (RR) are in the zone of potential bias or interest in the American College of Obstetricians and Gynecologists (ACOG) practice bulletins (PB); (2) the likelihood that false alarms have been linked to recommendations; and (3) if there are differences in obstetric versus gynecological PB, vis-à-vis OR and RR in the zone of bias or interest.

Data sources: We reviewed all ACOG PBs published between May 1999 and March 2014.

Methods: Documents were searched for statements with mention of OR and RR. When the reported ORs fell between 0.33 and 3.0, it was categorized as "zone of potential bias"; if ORs fell outside the interval 0.25 to 4.0, it was "zone of potential interest." With RRs, the zones of bias and interest were 0.5 to 2.0 and 0.33 to 3.0, respectively.

Results: Of the 79 PBs reviewed, 22% (n = 17) had 44 statements with OR, with 41% (n = 18) of the ORs being in the zone of bias and 54% (n = 24) in the zone of potential interest. Overall, 84% of the ORs did not lead to an actual recommendation by ACOG. In 28% (n = 22) of PBs, there were 67 statements with RRs, with 58% (n = 39) of them being in the zone of bias and 28% (n = 19) were in the zone of interest. In 73% of the PBs the RR citations did not lead to any recommendations. Across the 79 PBs, ACOG made 733 recommendations, and among them only 1 and 2% were linked with ORs and RRs, respectively.

Conclusions: Over 70% of ACOGPBs did not cite OR and RR. To better understand the evidence, ACOG should increase citation of OR and RR; whenever applicable OR and RR should be part of graded recommendations.

Level of evidence: Level C.

Précis: Less than a third of ACOG practice bulletins mention odds ratios or relative risks and over 95% of recommendations in them remain unsubstantiated.