Quality assessment of economic evaluations in selected pharmacy, medical, and health economics journals

Abstract

Objective: To assess and compare the quality of economic studies in selected pharmacy, medical, and health economics journals.

Data sources: DICP The Annals of Pharmacotherapy, American Journal of Hospital Pharmacy, Hospital Pharmacy, New England Journal of Medicine, Medical Care, Journal of the American Medical Association, PharmacoEconomics, International Journal of Technology Assessment in Health Care, and Journal of Health Economics using MEDLINE, EMBASE, and International Pharmaceutical Abstracts. Search terms included "economic," "cost," and "cost analysis."

Study selection: Reviewers appraised abstracts to identify original research published during 1989-1993 comparing costs and outcomes between drugs, treatments, and/or services. Initially, 123 articles met criteria; 16 were inappropriate, 17 were randomized out, and 90 (73%) were used (30/group).

Data extraction: Quality was assessed using a 13-item checklist. Interrater reliability was 0.91 (p < 0.05) for 9 raters, test-retest reliability was 0.94 (p < 0.001).

Data synthesis: A 2-way ANOVA, with overall quality scores as a dependent variable with journal type and year as independent variables, was significant (F = 2.79, p = 0.002, r2 = 0.34), with no significant interaction (F = 0.71, p = 0.68) or time effect (F = 0.70, p = 0.60). Journal types differed; pharmacy journals scored significantly lower (chi 2 = 53.89, df = 2, p < 0.001). Items rated adequate (i.e., correct or acceptable) increased over time (chi 2 = 21.18, df = 4, p < 0.001). Ethical issues and study perspective most needed improvement.

Conclusions: Article quality for all journal types increased over time nonsignificantly; health economics journals scored highest, then medical journals, with pharmacy journals significantly lower (and having the highest standard deviation). We recommend that authors and reviewers pay closer attention to study perspective and ethical implications.