The SCOPE framework - implementing ideals of responsible research assessment

Abstract

Background: Research and researchers are heavily evaluated, and over the past decade it has become widely acknowledged that the consequences of evaluating the research enterprise and particularly individual researchers are considerable. This has resulted in the publishing of several guidelines and principles to support moving towards more responsible research assessment (RRA). To ensure that research evaluation is meaningful, responsible, and effective the International Network of Research Management Societies (INORMS) Research Evaluation Group created the SCOPE framework enabling evaluators to deliver on existing principles of RRA. SCOPE bridges the gap between principles and their implementation by providing a structured five-stage framework by which evaluations can be designed and implemented, as well as evaluated.

Methods: SCOPE is a step-by-step process designed to help plan, design, and conduct research evaluations as well as check effectiveness of existing evaluations. In this article, four case studies are presented to show how SCOPE has been used in practice to provide value-based research evaluation.

Results: This article situates SCOPE within the international work towards more meaningful and robust research evaluation practices and shows through the four case studies how it can be used by different organisations to develop evaluations at different levels of granularity and in different settings.

Conclusions: The article demonstrates that the SCOPE framework is rooted firmly in the existing literature. In addition, it is argued that it does not simply translate existing principles of RRA into practice, but provides additional considerations not always addressed in existing RRA principles and practices thus playing a specific role in the delivery of RRA. Furthermore, the use cases show the value of SCOPE across a range of settings, including different institutional types, sizes, and missions.

Keywords: Research evaluation; SCOPE; evaluation framework; responsible research assessment.

Figure 1.. The SCOPE Framework (from The SCOPE Framework: a five-stage process for evaluating responsibly: https://doi.org/10.26188/21919527.v1).

Figure 2.. SCOPE ‘Context’ Matrix defining where assessments may have a low/medium/high impact on the assessed entity (from The SCOPE Framework: a five-stage process for evaluating responsibly: https://doi.org/10.26188/21919527.v1).

Figure 3.. Spectra used to identify stakeholder appetites for different evaluation variables (from INORMS SCOPE Case Study: The UK Higher Education Funding Bodies - Developing a new national research evaluation system: https://inorms.net/wp-content/uploads/2021/11/inorms-scope-case-study-uk-he-funding-bodies-final.pdf).