Time-to-update of systematic reviews relative to the availability of new evidence

Abstract

Background: A number of methods for deciding when a systematic review should be updated have been proposed, yet little is known about whether systematic reviews are updated more quickly when new evidence becomes available. Our aim was to examine the timing of systematic review updates relative to the availability of new evidence.

Methods: We performed a retrospective analysis of the update timing of systematic reviews published in the Cochrane Database of Systematic Reviews in 2010 relative to the availability of new trial evidence. We compared the update timing of systematic reviews with and without signals defined by the completion or publication of studies that were included in the updates.

Results: We found 43% (293/682) systematic reviews were updated before June 2017, of which 204 included an updated primary outcome meta-analysis (median update time 35.4 months; IQR 25.5-54.0), 38% (77/204) added new trials, and 4% (8/204) reported a change in conclusion. In the 171 systematic reviews with reconcilable trial reporting information, we did not find a clear difference in update timing (p = 0.05) between the 15 systematic reviews with a publication signal (median 25.3 months; IQR 15.3-43.5) and the 156 systematic reviews without a publication signal (median 34.4 months; IQR 25.1-52.2). In the 145 systematic reviews with reconcilable trial completion information, we did not find a difference in update timing (p = 0.33) between the 15 systematic reviews with a trial completion signal (median 26.0 months; IQR 19.3-49.5) and the 130 systematic reviews without a trial completion signal (median 32.4 months; IQR 24.1 to 46.0).

Conclusion: A minority of 2010 Cochrane reviews were updated before June 2017 to incorporate evidence from new primary studies, and very few updates led to a change in conclusion. We did not find clear evidence that updates were undertaken faster when new evidence was made available. New approaches for finding early signals that a systematic review conclusion is at risk of change may be useful in allocated resources to the updating of systematic reviews.

Keywords: Clinical trial registries; Evidence synthesis; Systematic reviews; Updating systematic reviews.

Fig. 1

From 773 articles published in 2010 in the Cochrane Database of Systematic Reviews, 293 were included in the analysis

Fig. 2

Update timing in systematic reviews published in Cochrane Database of Systematic Reviews in 2010 (SRs = systematic reviews, MAs = meta-analyses)

Fig. 3

The completeness of 60 systematic reviews that had new trials added to a primary outcome meta-analysis in an update. The median completeness is represented for each systematic review from its publication date to the search date of its update (length of follow-up is marked), interquartile range (dark grey), and range (light grey). Individual completeness values at the search date of an update are illustrated for those with changes in conclusion (black dots) and no change in conclusion (grey dots)

Fig. 4

Time to update for a 204 systematic reviews with a primary outcome meta-analysis (green) compared to 89 systematic reviews without a primary outcome meta-analysis (orange), b 15 systematic reviews with a publication signal (green) compared to 156 systematic reviews without a publication signal (orange), and c 15 systematic reviews with a trial completion signal (green) compared to 130 systematic reviews without a trial completion signal (orange). Shaded regions indicate the 95% confidence interval