Factors Associated with the Quality and Transparency of National Guidelines: A Mixed-Methods Study

Abstract

We assessed the methodological quality and transparency of all the national clinical practice guidelines that were published in Croatia up until 2017 and explored the factors associated with their quality rating. An in-depth quantitative and qualitative analysis was performed using rigorous methodology. We evaluated the guidelines using a validated AGREE II instrument with four raters; we used multiple linear regressions to identify the predictors of quality; and two focus groups, including guideline developers, to further explore the guideline development process. The majority of the guidelines (N = 74) were developed by medical societies. The guidelines' quality was rated low: the median standardized AGREE II score was low, 36% (IQR 28-42), and so were the overall-assessments. The aspects of the guidelines that were rated best were the "clarity of presentation" and the "scope and purpose" (median ≥ 59%); however, the other four domains received very low scores (15-33%). Overall, the guideline quality did not improve over time. The guidelines that were developed by medical societies scored significantly worse than those developed by governmental, or unofficial working groups (12-43% per domain). In focus group discussions, inadequate methodology, a lack of implementation systems in place, a lack of awareness about editorial independence, and broader expertise/perspectives in working groups were identified as factors behind the low scores. The factors identified as affecting the quality of the national guidelines may help stakeholders who are developing interventions and education programs aimed at improving guideline quality worldwide.

Keywords: clinical practice guidelines; education; focus group; guideline development; knowledge; methodological quality; national guidelines; public health.

Figure 1

The authorship landscape of 74 Croatian clinical practice guidelines (649 unique authors, median—11 authors per guideline). Authors are identified by specialty, followed by an index letter after the comma that uniquely identifies a person. The size of the circle reflects the number of CPGs (≥2) published by this person.

Figure 2

Standardized domain score distribution across six AGREE II domains: (A) among the Croatian clinical practice guidelines, N = 74. Outliers and extreme outliers are represented by circles and asterixis, respectively; (B) among countries that used AGREE II to evaluate national guidelines. The period studied and the country of origin are indicated in the legend of the figure. Japan was the only country reported to have high-quality guidelines [24], whereas all others were classified as those with low-quality guidelines [20,21,22,33,34,35].

Figure 3

Croatian guidelines’ AGREE II scores in the domains “stakeholder involvement” and “editorial independence” increased over time. Shown here are the individual scores, as well as the simple regression line and its accompanying 95% prediction interval.