Problems with the interpretation of pharmacoeconomic analyses: a review of submissions to the Australian Pharmaceutical Benefits Scheme

Abstract

Context: Pharmacoeconomic analyses are being used increasingly as the basis for reimbursement of the costs of new drugs. Reports of these analyses are often published in peer-reviewed journals. However, the analyses are complex and difficult to evaluate.

Objective: To describe the nature of problems encountered in the evaluation and interpretation of pharmacoeconomic analyses used as a basis for reimbursement decisions.

Data sources: All major submissions to the Department of Health and Aged Care (DHAC) by the pharmaceutical industry for funding made under the Australian Pharmaceutical Benefits Scheme. Specifically, the DHAC's database of submissions that were received between January 1994 and December 1997 were reviewed.

Study selection: Of a total of 326 submissions, 218 had serious problems of interpretation and were included in the analysis. The nature of the serious problems reviewed were classified as estimates of comparative clinical efficacy, comparator issues, modeling issues, and calculation errors.

Data extraction: All submissions in the DHAC's database were reviewed and data were extracted if both the DHAC evaluators and technical subcommittee considered problems to have a significant bearing on the decisions of the parent committee.

Data synthesis: Of a total of 326 submissions, 218 (67%) had significant problems and 31 had more than 1 problem. Of the 249 problems identified, 154 (62%) related to uncertainty in the estimates of comparative clinical efficacy, and 71 (28.5%) related to modeling issues, which included clinical assumptions or cost estimates, used in the construction of the economic models. There were 15 instances of disagreement over the choice of comparator, and serious calculation errors were found on 9 occasions. Overall, 159 problems (64%) were considered to be avoidable.

Conclusions: Significant problems were identified in these pharmacoeconomic analyses. The intensive evaluation process used in the Australian Pharmaceutical Benefits Scheme allowed for identification and correction of pharmacoecomomic analysis problems, but the resources that are required may be beyond the capacity of many organizations, including peer-reviewed journals.