Approaches to enabling rapid evaluation of innovations in health and social care: a scoping review of evidence from high-income countries

Abstract

Objective: The COVID-19 pandemic increased the demand for rapid evaluation of innovation in health and social care. Assessment of rapid methodologies is lacking although challenges in ensuring rigour and effective use of resources are known. We mapped reports of rapid evaluations of health and social care innovations, categorised different approaches to rapid evaluation, explored comparative benefits of rapid evaluation, and identified knowledge gaps.

Design: Scoping review.

Data sources: MEDLINE, EMBASE and Health Management Information Consortium (HMIC) databases were searched through 13 September 2022.

Eligibility criteria for selecting studies: We included publications reporting primary research or methods for rapid evaluation of interventions or services in health and social care in high-income countries.

Data extraction and synthesis: Two reviewers developed and piloted a data extraction form. One reviewer extracted data, a second reviewer checked 10% of the studies; disagreements and uncertainty were resolved through consensus. We used narrative synthesis to map different approaches to conducting rapid evaluation.

Results: We identified 16 759 records and included 162 which met inclusion criteria.We identified four main approaches for rapid evaluation: (1) Using methodology designed specifically for rapid evaluation; (2) Increasing rapidity by doing less or using less time-intensive methodology; (3) Using alternative technologies and/or data to increase speed of existing evaluation method; (4) Adapting part of non-rapid evaluation.The COVID-19 pandemic resulted in an increase in publications and some limited changes in identified methods. We found little research comparing rapid and non-rapid evaluation.

Conclusions: We found a lack of clarity about what 'rapid evaluation' means but identified some useful preliminary categories. There is a need for clarity and consistency about what constitutes rapid evaluation; consistent terminology in reporting evaluations as rapid; development of specific methodologies for making evaluation more rapid; and assessment of advantages and disadvantages of rapid methodology in terms of rigour, cost and impact.

Keywords: Organisation of health services; Quality in health care; STATISTICS & RESEARCH METHODS.

Figure 1

Preferred Reporting Items for Systematic Review and Meta-Analysis diagram showing study flow for the review. LMIC; low-income and middle-income country.