Usability of a barcode scanning system as a means of data entry on a PDA for self-report health outcome questionnaires: a pilot study in individuals over 60 years of age

Abstract

Background: Throughout the medical and paramedical professions, self-report health status questionnaires are used to gather patient-reported outcome measures. The objective of this pilot study was to evaluate in individuals over 60 years of age the usability of a PDA-based barcode scanning system with a text-to-speech synthesizer to collect data electronically from self-report health outcome questionnaires.

Methods: Usability of the system was tested on a sample of 24 community-living older adults (7 men, 17 women) ranging in age from 63 to 93 years. After receiving a brief demonstration on the use of the barcode scanner, participants were randomly assigned to complete two sets of 16 questions using the bar code wand scanner for one set and a pen for the other. Usability was assessed using directed interviews with a usability questionnaire and performance-based metrics (task times, errors, sources of errors).

Results: Overall, participants found barcode scanning easy to learn, easy to use, and pleasant. Participants were marginally faster in completing the 16 survey questions when using pen entry (20/24 participants). The mean response time with the barcode scanner was 31 seconds longer than traditional pen entry for a subset of 16 questions (p = 0.001). The responsiveness of the scanning system, expressed as first scan success rate, was less than perfect, with approximately one-third of first scans requiring a rescan to successfully capture the data entry. The responsiveness of the system can be explained by a combination of factors such as the location of the scanning errors, the type of barcode used as an answer field in the paper version, and the optical characteristics of the barcode scanner.

Conclusion: The results presented in this study offer insights regarding the feasibility, usability and effectiveness of using a barcode scanner with older adults as an electronic data entry method on a PDA. While participants in this study found their experience with the barcode scanning system enjoyable and learned to become proficient in its use, the responsiveness of the system constitutes a barrier to wide-scale use of such a system. Optimizing the graphical presentation of the information on paper should significantly increase the system's responsiveness.

Figure 1

Overview of proposed system to collect outcome data electronically. A mobile computer (top left panel) accepting flash card adapters and running the Palm OS is connected to a barcode wand scanner to input outcome data. The survey questions and their answer fields are presented on paper (top right panel). Barcodes are associated with specific answer fields. The layout can be adjusted (type face, presentation of information etc.). Once collected electronically on the mobile computer, the information can be transferred to a PC database.

Figure 2

Individual results comparing the total time to complete task (TTCT). A) Pen entry (PE) on paper vs. first round of bar code entry (BCE) after a brief tutorial and practice session (n = 24). B) First round of bar code entry after brief tutorial and practice session vs. second round of data entry using barcode entry for participants who chose to continue with the barcode system (n = 18). Mean time and standard deviation for TTCT with BCE was 235 ± 74 seconds vs. 204 ± 61 seconds for TTCT with PE in the first round of data entry. For subjects who completed 2 rounds of data entry using the barcode system, mean time and standard deviation for TTCT with BCE on the first round was 219 ± 37 seconds vs. 191 ± 25 seconds for TTCT with BCE in the second round of data entry.

Figure 3

Scan success rates when using the bar code entry system during the tutorial/practice (T), first round of bar code entry (BCE1) and second round of bar code entry (BCE2). A) Mean and standard deviation of first scan success rate expressed as a % of the questions answered across participants (n = 24). B) Mean and standard deviation of second scan success rate for unsuccessful first scan (expressed as a % of the questions answered) across participants (n = 24).

Figure 4

Scan errors. A) Frequency distribution; and B) Location of first scan errors on paper. Most first scan errors occurred when scanning barcodes associated with answers positioned on the far left of the paper.

Figure 5

Subjective evaluation of barcode scanning experience as measured by usability questionnaire (see Appendix 1) administered after tutorial/practice, first round of data entry, and second round of data entry. Frequency distribution (n _total = 24) of mean scores on: A) Learnability of using a barcode scanner for data entry; B) Satisfaction after using the barcode scanner; and C) Experience with bar code scanner vs. pen entry. A mean score higher than 60% indicates agreement with statements.