Evaluation of guideline quality using GEM-Q

Abstract

A variety of rating instruments that evaluate the quality of practice guidelines have been published. Application of these instruments can be difficult and time-consuming. In a literature review, we identified two evaluation instruments that are comprehensive, have clearly defined constructs, and have undergone validation/testing-the Guidelines Quality Assessment Questionnaire (GQAQ) and the Appraisal Instrument for Clinical Guidelines (AICG). Overall, the AICG is more comprehensive. The AICG addresses the implementability of a guideline, which is not evaluated by the GQAQ. However, the GQAQ is more amenable to computerization. GEM-Q is a Guideline Elements Model (GEM)-derived application intended to facilitate automated evaluation of guideline quality using one of the published instruments. To develop GEM-Q, various items in the GQAQ were mapped to corresponding elements in the GEM hierarchy and a customized XSL stylesheet was designed based on this mapping. GEM-Q selectively extracts text components of the guideline relevant to quality evaluation and displays the results in HTML format. GEM-Q was applied to a set of six guidelines to test its reliability. It ranked two guidelines as of "good" quality, two as "intermediate", and two as "poor". In all six instances, GEM-Q ranked guidelines in the same order of quality as the experts who validated the GQAQ. This work demonstrates the feasibility of developing an application to facilitate automated guideline quality evaluation.