Assessing the performance of indicators during their life cycle: the mixed QUID method

Abstract

Background: Quality indicators (QI) are mandatory in French hospitals. After a decade of use, the Ministry of Health set up an expert workgroup to enhance informed decision-making regarding currently used national QI, i.e. to propose a decision of withdrawing, revising or continuing their use. We report the development of an integrated method for a comprehensive appraisal of quality/safety indicators (QI) during their life cycle, for three purposes, quality improvement, public disclosure and regulation purposes. The method was tested on 10 national QI on use for up to 10 years to identify operational issues.

Methods: A modified Delphi technique to select relevant criteria and a development of a mixed evaluation method by the workgroup. A 'real-life' test on 10 national QI.

Results: Twelve criteria were selected for the appraisal of QI used for regulation goals, 11 were selected for hospital improvement and seven for public disclosure. The perceived feasibility and relevance were studied including hospital workers, patients and health authorities professionals; the scientific soundness of the indicator development phase was reviewed by analyzing reference documents; the metrological performance (limited to the discriminatory power and dynamics of change during the life cycle dimensions) was analyzed on the national datasets.Applied to the 10 QI, the workgroup proposed to withdraw four of them and to modify or suspend the six others.

Conclusions: The value of the method was supported by the clear-cut conclusions and endorsement of the proposed decisions by the health authorities.

Keywords: evaluation methodology; health care quality assessment; healthcare; methods; quality indicators.