Rapid evaluation methods (REM) of health services performance: methodological observations

Abstract

The rapid evaluation method (REM) was developed by WHO in order to assess the performance and quality of health care services, identify operational problems, and assist in taking managerial action. It was tested in five developing countries (Botswana, Madagascar, Papua New Guinea, Uganda and Zambia) between 1988 and 1991. REM consists of a set of observation- and survey-based diagnostic activities, carried out mainly in health care facilities. The article describes the various steps of REM, methodological issues such as setting objectives and using an issue-information matrix, preparation of survey instruments, use of computer software (Epi Info), data quality control, fieldwork, and the use of data to produce useful information for decision-makers. REM aims at bringing prompt and relevant information to planners and decision-makers who need it for a specific purpose. In the present examples, REM provided information for preparing a programme proposal for external funding, for establishing baseline data for a situation analysis, and for assessing staff performance after extensive training in order to improve the curriculum.

PIP: The rapid evaluation method (REM) is a health management tool aimed at bringing prompt an relevant information to planners and decision makers. The method was developed by WHO's Family Health Division and was field tested in mother and child health care and family planning facilities in Botswana, Madagascar, Papua New Guinea, Uganda, and Zambia between 1988 and 1991. The purpose of this paper is to acquaint the reader with the basic components of the method and methodological problems that arise. Each of the data collection instruments employed is described: clinic exit interviews, health staff interviews, observation of task performance, community and staff focus group discussions, review of clinical records, checking of facilities and equipment and supplies, and household interviews. REM was developed because of the apparent problems of too much paperwork and insufficient time for useful analysis and fears of the expense of surveys. There was a need for a quick, accurate, and economical method of evaluation of facilities and client satisfaction. Reference is made to other REM approaches and a review of REM methods. A basic requirement is the involvement of national program managers in the control, implementation, and application of the design; outside consultants furnish information on methods, formats, and analytical techniques as a complement to national efforts. THe first action taken is to define the objectives and specify the topics and issues of concern. A core group is assigned responsibility for REM. The action plan involves the objectives, information desired, sources for information, schedule of activities, logistical arrangements, and budget preparation. The level of detail of the information desired is defined by the core group and its objectives. A matrix of information is developed which includes information requested and sources; an example is provided. The results of the 5-county evaluation revealed methodological concern about the identification of issues and pretesting of instruments, data quality control, advance notice and planning of fieldwork, the need for analysts familiar with Epi Info Software, analysis of quantitative data first and training of leaders for focus groups, and the need for basic results in 7-10 days and a draft report in several weeks.