AR DriveSim: An Immersive Driving Simulator for Augmented Reality Head-Up Display Research

Abstract

Optical see-through automotive head-up displays (HUDs) are a form of augmented reality (AR) that is quickly gaining penetration into the consumer market. Despite increasing adoption, demand, and competition among manufacturers to deliver higher quality HUDs with increased fields of view, little work has been done to understand how best to design and assess AR HUD user interfaces, and how to quantify their effects on driver behavior, performance, and ultimately safety. This paper reports on a novel, low-cost, immersive driving simulator created using a myriad of custom hardware and software technologies specifically to examine basic and applied research questions related to AR HUDs usage when driving. We describe our experiences developing simulator hardware and software and detail a user study that examines driver performance, visual attention, and preferences using two AR navigation interfaces. Results suggest that conformal AR graphics may not be inherently better than other HUD interfaces. We include lessons learned from our simulator development experiences, results of the user study and conclude with limitations and future work.

Keywords: augmented reality; conformal graphics; driving simulator; head-up display; human machine interface.

Figure 1

A bird's eye view of the Mini Cooper half cab with participant and experimenter. While the work presented herein focuses on CG-based driving capabilities, the testbed also supports alternative forms of driving studies (e.g., video-based).

Figure 2

The logical arrangements of 7 AR DriveSim screens with resolution and physical connections noted (A); experimenter's control station (B), and an annotated view from inside the Mini Cooper cab (C). Note that the AR HUD and Center Stack displays are connected to a separate computer dedicated to UI presentation.

Figure 3

The major computing components of our AR DriveSim communicate via UDP (yellow). A set of off-the-shelf and custom microcontrollers (brown) pass driving control inputs read from CAN bus (green) and other sensors to the AR DriveSim computer (orange). A control board (brown) further manages a DC motor to provide force feedback to the steering wheel. A separate computer (blue) renders 3D graphics on an AR HUD by synchronizing its virtual camera position in real-time with the AR DriveSim computer. A set of experimenter controls (black) assist in coordinating experiments.

Figure 4

A participant's view as they calibrate the AR HUD 3D graphics by viewing carefully placed 3D shapes and perspective lines overlaid onto a static road scene with known geometry.

Figure 5

The user study examined two HUD display conditions: a conformal arrow integrated into road scene (top), and, a screen-fixed arrow which filled as drivers approached turns (bottom). For each, the initial state of the graphic appeared at 392 feet from the interaction (left panels), and disappeared after participants traversed intersections (right panels).

Figure 6

Participants rated the NASA-TLX sub-scores on a scale of 0 (low demand) to 100 (high demand). The average of the sub-scores comprised the Raw TLX score.

Figure 7

Participants rated the following statements on a scale of 0 (strongly agree) to 100 (strongly disagree): (1) “I did not find this interface distracting.” (2) “Using this interface had a positive impact on my driving.” (3) “It was easy to navigate while using this interface.” (4) “I trusted the information on this interface.” and (5) “The interface was easy to view”.

Figure 8

The percentage of fixations allocated to each AOI differed between Conformal and Screen-fixed displays. While there is no ideal allocation across AOIs, it is interesting to note that the percentages vary, particularly between percentage of time spent looking at and around the HUD Graphic.