Validity of performance indicators for assessing prescribing quality: the case of asthma

Abstract

Objectives: The aim of this study was to assess the concurrent validity between the identification of sub-optimal treatment based on clinical information and computer generated indicators. Indicators that are associated with sub-optimal treatment in one of the four steps of asthma management were assessed.

Design: The ability of each indicator to identify patients with sub-optimal asthma treatment from computerised general practitioner (GP) prescription records was assessed by comparing them with the results of an individual patient assessment using clinical data.

Setting: Chronic asthma patients ( n=146) registered with 16 Dutch GPs.

Main measures: The sensitivity and positive predictive value (PPV) of each performance indicator was determined.

Results: The step-1 indicator, focusing on patients not prescribed a short-acting beta-agonist, had an acceptable sensitivity (0.86), but a low PPV (0.52). The two step-2 indicators, targeting under-prescription of inhaled corticosteroids, had sensitivities of 0.74 and 0.37 and PPVs of 0.46 and 0.71, respectively. The step-3 indicator, which targeted under-dosing of inhaled corticosteroids, had a sensitivity of 0.07 and a PPV of 0.2. The fourth indicator, focusing on under-prescription of long-acting beta-agonists, could not be validated due to inadequate numbers of patients with severe asthma in our study sample.

Discussion: None of the indicators investigated was considered valid for assessing prescriber performance, despite having good face and content validity. Performance indicators that have not been validated can only provide a broad-brush approach for assessing prescribing quality and should be used with extreme caution.