Evaluation of Electronic and Paper-Pen Data Capturing Tools for Data Quality in a Public Health Survey in a Health and Demographic Surveillance Site, Ethiopia: Randomized Controlled Crossover Health Care Information Technology Evaluation

Abstract

Background: Periodic demographic health surveillance and surveys are the main sources of health information in developing countries. Conducting a survey requires extensive use of paper-pen and manual work and lengthy processes to generate the required information. Despite the rise of popularity in using electronic data collection systems to alleviate the problems, sufficient evidence is not available to support the use of electronic data capture (EDC) tools in interviewer-administered data collection processes.

Objective: This study aimed to compare data quality parameters in the data collected using mobile electronic and standard paper-based data capture tools in one of the health and demographic surveillance sites in northwest Ethiopia.

Methods: A randomized controlled crossover health care information technology evaluation was conducted from May 10, 2016, to June 3, 2016, in a demographic and surveillance site. A total of 12 interviewers, as 2 individuals (one of them with a tablet computer and the other with a paper-based questionnaire) in 6 groups were assigned in the 6 towns of the surveillance premises. Data collectors switched the data collection method based on computer-generated random order. Data were cleaned using a MySQL program and transferred to SPSS (IBM SPSS Statistics for Windows, Version 24.0) and R statistical software (R version 3.4.3, the R Foundation for Statistical Computing Platform) for analysis. Descriptive and mixed ordinal logistic analyses were employed. The qualitative interview audio record from the system users was transcribed, coded, categorized, and linked to the International Organization for Standardization 9241-part 10 dialogue principles for system usability. The usability of this open data kit-based system was assessed using quantitative System Usability Scale (SUS) and matching of qualitative data with the isometric dialogue principles.

Results: From the submitted 1246 complete records of questionnaires in each tool, 41.89% (522/1246) of the paper and pen data capture (PPDC) and 30.89% (385/1246) of the EDC tool questionnaires had one or more types of data quality errors. The overall error rates were 1.67% and 0.60% for PPDC and EDC, respectively. The chances of more errors on the PPDC tool were multiplied by 1.015 for each additional question in the interview compared with EDC. The SUS score of the data collectors was 85.6. In the qualitative data response mapping, EDC had more positive suitability of task responses with few error tolerance characteristics.

Conclusions: EDC possessed significantly better data quality and efficiency compared with PPDC, explained with fewer errors, instant data submission, and easy handling. The EDC proved to be a usable data collection tool in the rural study setting. Implementation organization needs to consider consistent power source, decent internet connection, standby technical support, and security assurance for the mobile device users for planning full-fledged implementation and integration of the system in the surveillance site.

Keywords: Ethiopia; data collection; data quality; mHealth; maternal health; public health; surveillance; survey; tablet computer.

Figure 1

Screenshot examples of the type of questions; multiple choices (A), number (B), single select (C), and date (D) presented in the electronic data capture tool used for the survey in Dabat Demographic and Health Surveillance site in June 2016, northwest Ethiopia. Translation of the Screen: (A) 217: Which of the signs of pregnancy complications or danger sign of pregnancy have you encountered during postnatal period? bleeding, fever, vaginal gush of fluid, incontinency, and other (describe); (B) 204: Birth weight for the last child (if it was measured); (C) 213: Where did you give birth for your last child? home, hospital, health center, health post, and other (please mention it here); and (D) 218: When was the last child delivered?

Figure 2

Frequency of error comparison among the electronic data capture (EDC) tools using tablet computer and paper and pen data capture (PPDC) tools during a survey in the demographic survey site in 2016, Dabat, northwest Ethiopia.

Figure 3

Mean values of the overall error rates trend of electronics data capture tool (EDC) using tablet and paper and pen data capture (PPDC) tools used during the survey in the demographic survey site in 2016, Dabat, northwest Ethiopia.

Figure 4

Usability response for negatively articulated questions in the System Usability Scale (SUS) used during a survey in the demographic survey site in 2016, Dabat, northwest Ethiopia.

Figure 5

Usability questionnaire response for positively articulated questions in the System Usability Scale (SUS) used during a survey in a demographic survey site in 2016, Dabat, northwest Ethiopia.