Research Integrity in Guidelines and evIDence synthesis (RIGID): a framework for assessing research integrity in guideline development and evidence synthesis

Abstract

Background: Clinical guidelines rely on sound evidence to underpin recommendations for patient care. Compromised research integrity can erode public trust and the credibility of the scientific enterprise, with potential harm to patients. Despite increased recognition of integrity concerns in scientific literature, there are no processes or guidance for incorporating integrity assessments into evidence-based guideline development or evidence synthesis more broadly.

Methods: In response to this crucial gap, we developed the Research Integrity in Guidelines and evIDence synthesis (RIGID) framework. Co-developed with international input from 80 multidisciplinary experts, and consumers, the RIGID framework and accompanying checklist provide an innovative and transparent six-step approach to assess the integrity of studies during the synthesis of evidence, including in the development of clinical guidelines. Central to the framework is an integrity committee, responsible for objective assessments and allocations, with constructive author engagement.

Findings: The six key steps of the RIGID framework are described, as follows: (1) Review: standard systematic review processes are followed, in line with approved evidence synthesis methodologies; (2) Exclude: studies which have been retracted are excluded, and those with expressions of concern are flagged for further evaluation; (3) Assess: remaining studies are assessed for integrity using an appropriate tool and allocated an initial integrity risk rating of low, moderate or high risk for integrity concerns; (4) Discuss: integrity assessment results are discussed among integrity committee members with votes to determine final integrity risk rating allocations for each study; (5) Establish contact: low risk studies are included without author contact, whereas authors of studies ranked as moderate or high risk are contacted for clarification; (6) Reassess: studies are reassessed for inclusion using the RIGID author response algorithm (reclassified as 'included' where authors have provided a satisfactory response, 'awaiting classification' where authors have engaged but time is needed to address concerns, or 'not included' where authors have not responded to contact attempts). An illustrative case study is presented, where these six steps of the RIGID framework were successfully implemented in an influential international guideline endorsed by 39 societies across six continents. Following implementation of the framework, 45 of the 101 originally identified studies (45%) were not included in the guideline.

Interpretation: Based on the latest literature and international expertise, the RIGID framework represents an important advancement in best practice standards for guideline development and evidence synthesis. Using this resource, guideline developers, policy-makers, clinicians and scientists are better positioned to navigate the currently precarious research landscape to ensure evidence synthesis and subsequent clinical recommendations prioritize patient care and preserve the sanctity of scientific endeavors.

Funding: This study received no specific funding. The guideline in which it was piloted was supported by the Australian National Health and Medical Research Council (NHMRC) for guideline development through the Centre of Research Excellence (CRE) in Women's Health in Reproductive Life (CRE-WHiRL) (APP1171592) and the CRE in Polycystic Ovary Syndrome (CRE-PCOS) (APP1078444) led by Monash University, Australia, and partner societies: the American Society for Reproductive Medicine (ASRM), the US Endocrine Society (ENDO), the European Society of Endocrinology (ESE) and the European Society of Human Reproduction and Embryology (ESHRE).

Keywords: Clinical practice recommendations; Evidence synthesis; Evidence-based guidelines; Guideline development; Research integrity; Retraction.

Fig. 1

Research Integrity in Guidelines and EvIDence Synthesis (RIGID) Framework: a process for incorporating research integrity assessments into evidence synthesis and guideline development. GDG, guideline development group; GRADE, Grading of Recommendations, Assessment, Development and Evaluations; TRACT, Trustworthiness in Randomised Controlled Trials. ∗meta-analysis should not be performed until all authors have been contacted and, time-permitting, where relevant studies have been re-classified. A consistent timeframe for responses to email contact should be applied, with a suggested minimum of two weeks for initial engagement.

Fig. 2

Incorporating integrity assessments into existing evidence synthesis processes.

Fig. 3

Algorithm based on author(s) engagement: the final step of the Research Integrity for Guidelines and EvIDence Synthesis (RIGID) framework. Author responses, or lack thereof, determine the subsequent categorization of potentially eligible studies, with inclusion determined by whether results of a study can be reliably used to inform evidence synthesis. ∗supporting evidence may include ethics applications, unregistered date-stamped protocols, sharing of individual participant data or other documentation as relevant, to be decided on a case-by-case basis by the integrity committee and documented transparently in publications or accompanying reports.

Fig. 4

Research Integrity in Guidelines and evIDence synthesis (RIGID) in Action: a case study of the RIGID framework when piloted in the context of a large international evidence-based guideline, illustrating application of the six RIGID steps. The international guideline development group who provided input and endorsed the framework are listed in the acknowledgements. MOD, moderate; RIGID, Research Integrity in Guidelines and evIDence synthesis.