From the Lab to People's Home: Lessons from Accessing Blind Participants' Interactions via Smart Glasses in Remote Studies

Abstract

Researchers have adopted remote methods, such as online surveys and video conferencing, to overcome challenges in conducting in-person usability testing, such as participation, user representation, and safety. However, remote user evaluation on hardware testbeds is limited, especially for blind participants, as such methods restrict access to observations of user interactions. We employ smart glasses in usability testing with blind people and share our lessons from a case study conducted in blind participants' homes (N = 12), where the experimenter can access participants' activities via dual video conferencing: a third-person view via a laptop camera and a first-person view via smart glasses worn by the participant. We show that smart glasses hold potential for observing participants' interactions with smartphone testbeds remotely; on average 58.7% of the interactions were fully captured via the first-person view compared to 3.7% via the third-person. However, this gain is not uniform across participants as it is susceptible to head movements orienting the ear towards a sound source, which highlights the need for a more inclusive camera form factor. We also share our lessons learned when it comes to dealing with lack of screen reader support in smart glasses, a rapidly draining battery, and Internet connectivity in remote studies with blind participants.

Keywords: blind people; remote method; smart glasses; user study.

Figure 1:

Dual video conferencing in our remote study design. While interacting with a testbed on a smartphone, a blind participant wearing smart glasses communicates with the experimenter through dual video conferencing. Two video streams are being sent to the experimenter: one from the participant’s smart glasses and the other from a laptop camera facing the participant on the same Zoom call.

Figure 2:

Two groups of study materials put into two different packages: a box (left) and a reusable shopping tote bag (right). Each package has unique texture. Participants were sent the study materials prior to their study session.

Figure 3:

A comparison between smart glasses and laptop cameras in terms of percentage of video frames across the tasks where overall participant interactions with the phone and stimuli were fully visible, partially visible, or not visible.

Figure 4:

A comparison between smart glasses and laptop camera in terms of percentage of video frames across the tasks where each participant interactions with the phone and stimuli were fully visible.

Figure 5:

Contrasting examples from the study showing how the two cameras capture complementary information.