EPC Response to IOM Standards for Systematic Reviews [Internet]

Excerpt

Background: The Institute of Medicine appointed an independent committee of experts to assess and to recommend a set of methodological standards that would assure objective, transparent, and scientifically valid systematic reviews of comparative effectiveness research. Following the release of these standards in March 2011, the EPC program established a collaborative process to comparatively examine the standards with respect to general EPC practice and guidance.

Purpose: The purpose of this process was to assess which elements of the IOM standards should be adopted into EPC methods guidance and how to best implement these changes, and which elements require further empirical evidence.

Methods: A two-phase approach is adopted, where in phase one, 13 EPC directors, in consultation with their respective staff, identified areas where general agreement exist and where further deliberation was necessary, and in phase two, workgroups, consisting of EPC investigators, were tasked to further deliberate and provide a disposition of each of those elements. These elements were categorized into one of four topic groups: program policies or procedures, protocol elements, searching/screening/reporting biases, and synthesis of evidence. Based on current practices and through discussions, four workgroups determined whether there was “agreement,” “agreement with modifications,” or “disagreement” for each element. Where there were modifications recommended, each workgroup provided a description of the differences between EPC practice or methods guidance and IOM guidelines, summarized the deliberative discussion, and made recommendations for further action.

Results: EPC directors identified 34 elements across the 21 standards that required remediation and assignment to one of the four workgroups. Workgroups described general agreement with the majority of these 34 elements in principle. There were three elements with which the EPCs were in disagreement, and were not recommended for routine practice as currently stated. Discussion on the remaining 31 elements pointed out inconsistency of practice and need for clearer guidance, the need for more empiric evidence in some cases, the difficulty in balancing benefits and the required resources for implementation of elements, and in some cases, specific suggestions on how to implement particular elements.

Conclusions/Recommendations: This process engaged the EPCs in a productive, collaborative evaluation and response to the work of the IOM committee for systematic review standards. Recommendations for further research or development, for updating of EPC guidance, and specific recommendations for practical implementation were itemized. Principally, this process will result in improvements in EPC practice.