User experience of home-based AbC-19 SARS-CoV-2 antibody rapid lateral flow immunoassay test

Abstract

The urgent need to scale up testing capacity during the COVID-19 pandemic has prompted the rapid development of point-of-care diagnostic tools such as lateral flow immunoassays (LFIA) for large-scale community-based rapid testing. However, studies of how the general public perform when using LFIA tests in different environmental settings are scarce. This user experience (UX) study of 264 participants in Northern Ireland aimed to gather a better understanding of how self-administered LFIA tests were performed by the general public at home. The UX performance was assessed via analysis of a post-test questionnaire including 30 polar questions and 11 7-point Likert scale questions, which covers the multidimensional aspects of UX in terms of ease of use, effectiveness, efficiency, accuracy and satisfaction. Results show that 96.6% of participants completed the test with an overall average UX score of 95.27% [95% confidence interval (CI) 92.71-97.83%], which suggests a good degree of user experience and effectiveness. Efficiency was assessed based on the use of physical resources and human support received, together with the mental effort of self-administering the test measured via NASA Task Load Index (TLX). The results for six TLX subscales show that the participants scored the test highest for mental demand and lowest for physical demand, but the average TLX score suggests that the general public have a relatively low level of mental workload when using LFIA self-testing at home. Five printed LFIA testing results (i.e. the 'simulated' results) were used as the ground truth to assess the participant's performance in interpreting the test results. The overall agreement (accuracy) was 80.63% [95% CI 75.21-86.05%] with a Kappa score 0.67 [95% CI 0.58-0.75] indicating substantial agreement. The users scored lower in confidence when interpreting test results that were weak positive cases (due to the relatively low signal intensity in the test-line) compared to strong positive cases. The end-users also found that the kit was easier to use than they expected (p < 0.001) and 231 of 264 (87.5%) reported that the test kit would meet their requirements if they needed an antibody testing kit. The overall findings provide an insight into the opportunities for improving the design of self-administered SARS-CoV-2 antibody testing kits for the general public and to inform protocols for future UX studies of LFIA rapid test kits.

Figure 1

Illustration of the LFIA testing device: (a) structure of the testing device showing the sample hole, viewing window with the control line and test line as an example of positive result; (b) an example of negative result, and (c) example of invalid (without C-line).

Figure 2

Examples of five printed test results that were designed for the study: (a) T1: positive; (b) T2: strong positive; (c) T3: negative; (d) T4: invalid, and (e) T5: weak positive.

Figure 3

The diagram for the study design.

Figure 4

(a) The histogram of age distribution for female and male participants; (b) percentage of participants in four education groups.

Figure 5

Summary of participants’ responses to polar questions survey together with p-values and effect size $\varphi$.

Figure 6

Average summative scores based on: (a) 25 UX questions; (b) 17 questions related to usability construct only. The error bars represent the standard errors (n = 264).

Figure 7

Summary for: (a) reasons for failure to complete the test based on 12 multiple choices; (b) areas of difficulty during the test based on 318 multiple choices.

Figure 8

Summary of number of participants’ rating in six TLX subscales.

Figure 9

Results of TLX and six subscales: (a) mean and standard deviation based on all participants (n = 264); (b) mean based on participants from four age groups (AGs); (c) mean based on participants from four education groups (EGs).

Figure 10

The counts for the 7-point Likert rating scores for question section Q12 (which helps to assess the comfort and acceptability).

Figure 11

The boxplot for: (a) scores from Q12b and Q12c to assess the user-perceived ease of using the kit after and before the test; (b) scores for Q12d (to assess the confidence in using the kit) from the users who completed and failed the test.