Claims of causality in health news: a randomised trial

Abstract

Background: Misleading news claims can be detrimental to public health. We aimed to improve the alignment between causal claims and evidence, without losing news interest (counter to assumptions that news is not interested in communicating caution).

Methods: We tested two interventions in press releases, which are the main sources for science and health news: (a) aligning the headlines and main causal claims with the underlying evidence (strong for experimental, cautious for correlational) and (b) inserting explicit statements/caveats about inferring causality. The 'participants' were press releases on health-related topics (N = 312; control = 89, claim alignment = 64, causality statement = 79, both = 80) from nine press offices (journals, universities, funders). Outcomes were news content (headlines, causal claims, caveats) in English-language international and national media (newspapers, websites, broadcast; N = 2257), news uptake (% press releases gaining news coverage) and feasibility (% press releases implementing cautious statements).

Results: News headlines showed better alignment to evidence when press releases were aligned (intention-to-treat analysis (ITT) 56% vs 52%, OR = 1.2 to 1.9; as-treated analysis (AT) 60% vs 32%, OR = 1.3 to 4.4). News claims also followed press releases, significant only for AT (ITT 62% vs 60%, OR = 0.7 to 1.6; AT, 67% vs 39%, OR = 1.4 to 5.7). The same was true for causality statements/caveats (ITT 15% vs 10%, OR = 0.9 to 2.6; AT 20% vs 0%, OR 16 to 156). There was no evidence of lost news uptake for press releases with aligned headlines and claims (ITT 55% vs 55%, OR = 0.7 to 1.3, AT 58% vs 60%, OR = 0.7 to 1.7), or causality statements/caveats (ITT 53% vs 56%, OR = 0.8 to 1.0, AT 66% vs 52%, OR = 1.3 to 2.7). Feasibility was demonstrated by a spontaneous increase in cautious headlines, claims and caveats in press releases compared to the pre-trial period (OR = 1.01 to 2.6, 1.3 to 3.4, 1.1 to 26, respectively).

Conclusions: News claims-even headlines-can become better aligned with evidence. Cautious claims and explicit caveats about correlational findings may penetrate into news without harming news interest. Findings from AT analysis are correlational and may not imply cause, although here the linking mechanism between press releases and news is known. ITT analysis was insensitive due to spontaneous adoption of interventions across conditions.

Trial registration: ISRCTN10492618 (20 August 2015).

Keywords: Media; Public health; Science communication; Science news.

Fig. 1

CONSORT diagram for the press releases (participants) in the trial. Inclusion criteria: participating press offices were asked to send each press release based on peer-reviewed research that was relevant to human health, broadly defined (all biomedical, psychological or lifestyle topics), where the press office was leading the press release (rather than collaborating on a release by another office outside the trial) and the academic authors consented (we used opt-out consent). Our focus was on observational and experimental studies. Observational studies included cross-sectional and longitudinal designs as well as meta-analyses and systematic reviews based solely on observational research. Experimental research included randomised controlled trials, other experiments and meta-analyses or systematic reviews based solely on experimental designs. Press releases on studies that could not be classified as experimental or observational (e.g. simulations and mixed methods reviews) were excluded

Fig. 2

a News follows the phrasing of the press release: In ITT and AT analysis, news headlines were more likely to align to evidence if the press release phrasing did so; and in the AT analysis, claims in the news text were also more likely to do so if the press release did so. The discrepancy between ITT and AT analyses was due to a high level of condition mixing (see text). b ITT and AT analyses both show no evidence of reduced news uptake for press releases whose headlines and main claims aligned to evidence (see also Additional file 1: Figure S4 for the average number of news per press release). Error bars are 95% CIs. For each bar, n reports total number of news (a) or press releases (b) in that analysis group (i.e. the denominator of the proportion that the bar displays; total n is lower for AT than ITT analysis, because AT was possible only for press releases with causal claims present in headlines or main claims)

Fig. 3

Feasibility and growing use of cautious headlines and main claims in observational research (error bars are 95% CIs). Feasibility is indicated by the increase in spontaneous use in pre-intervention (draft) press releases since the baseline period (2014/15). Final press releases showed small further increases in cautious wording following suggestions in the trial. For each bar, n reports the total number of press releases in that analysis group (i.e. the denominator of the proportion that the bar displays)

Fig. 4

Use of causality statements/caveats (error bars are 95%CIs). a ITT was insensitive to differences in news content; AT showed that 20% of news contained causality statements or caveats if the press release did, and almost never otherwise. b ITT shows no reduction of news uptake and AT shows an increase in news for press releases containing causality statements/caveats (see also Additional file 1: Figure S4 for average number of news per press release). c Feasibility is indicated by the increase in spontaneous caveats for observational research since the baseline period (2014/2015). Final press releases showed a further increase following suggestions in relevant trial conditions. For each bar, n reports total number of news (a) or press releases (b, c) in that analysis group (i.e. the denominator of the proportion that the bar displays)