Assessing the international use of health technology assessments: exploring the merits of different methods when applied to the National Institute of Health Research Health Technology Assessment (NIHR HTA) programme

Abstract

Objectives: This study presents findings from a study that explores the merits of different methods for assessing the international use of UK funded research by the National Institute of Health Research Health Technology Assessment (NIHR HTA) Programme.

Methods: The study adopted an exploratory approach and used three core methods: (i) Academic use was explored through bibliometric and citation analysis of the top ten most cited health technology assessment (HTA) reports. (ii) Internet use was assessed using Webtrends software to identify the proportion of international visits of the top ten most downloaded HTA reports from January 1, 2004 to June 30, 2010. (iii) International HTA use was assessed by searching the Center for Reviews and Dissemination (CRD) HTA database to explore the citation of NIHR HTA reports in reports by non-UK HTA agencies.

Results: Bibliometric analysis identified published output and international citations with 41 percent of the 549 journals citing NIHR HTA reports being based in the United States. Nine of ten most downloaded reports from the NIHR HTA Web site (www.hta.ac.uk) had in excess of 50 percent of visits outside the United Kingdom. Four of five selected NIHR HTA reports were cited in twenty-eight other HTA reports, eighteen of these outside the United Kingdom.

Conclusions: Assessing international use is important when exploring the uptake of research evidence. Methods used in identifying research impact, such as bibliometrics and Webtrends, are helpful in generating evidence of international use. HTA agencies should consider these techniques and international use when assessing the uptake of findings from research they undertake and/or commission.