Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2016 Mar;58(2):242-60.
doi: 10.1177/0018720815610271. Epub 2015 Oct 30.

The Effect of Incorrect Reliability Information on Expectations, Perceptions, and Use of Automation

Laura H Barg-Walkow  1 , Wendy A Rogers  2
Affiliations

The Effect of Incorrect Reliability Information on Expectations, Perceptions, and Use of Automation

Laura H Barg-Walkow et al. Hum Factors. 2016 Mar.

Abstract

Objective: We examined how providing artificially high or low statements about automation reliability affected expectations, perceptions, and use of automation over time.

Background: One common method of introducing automation is providing explicit statements about the automation's capabilities. Research is needed to understand how expectations from such introductions affect perceptions and use of automation.

Method: Explicit-statement introductions were manipulated to set higher-than (90%), same-as (75%), or lower-than (60%) levels of expectations in a dual-task scenario with 75% reliable automation. Two experiments were conducted to assess expectations, perceptions, compliance, reliance, and task performance over (a) 2 days and (b) 4 days.

Results: The baseline assessments showed initial expectations of automation reliability matched introduced levels of expectation. For the duration of each experiment, the lower-than groups' perceptions were lower than the actual automation reliability. However, the higher-than groups' perceptions were no different from actual automation reliability after Day 1 in either study. There were few differences between groups for automation use, which generally stayed the same or increased with experience using the system.

Conclusion: Introductory statements describing artificially low automation reliability have a long-lasting impact on perceptions about automation performance. Statements including incorrect automation reliability do not appear to affect use of automation.

Application: Introductions should be designed according to desired outcomes for expectations, perceptions, and use of the automation. Low expectations have long-lasting effects.

Keywords: automation; compliance and reliance; dependence; expectations; human–automation interaction; perceived reliability.

PubMed Disclaimer

Figures

Figure 1.
Figure 1.
Screen shot of the receiving packages task.
Figure 2.
Figure 2.
Screen shot of the dispatching trucks task. This represents what is seen when viewing the dispatching task (and not the receiving task) during an automation alert.
Figure 3.
Figure 3.
Experimental protocol. Days included in Experiment 1 are bolded, whereas days included in Experiment 2 are italicized. The same-as condition was only used in Experiment 1, as denoted by the dotted border. The strategies and perceptions survey was only administered in Experiment 2 on Day 4, as denoted by the dashed border.
Figure 4.
Figure 4.
Performance in total task points in Experiment 1. Error bars depict standard error of the mean.
Figure 5.
Figure 5.
Compliance per day in Experiment 1. The view of the lower-than group’s data is obstructed by the higher-than group’s data. The black line denotes actual automation performance (75% correct). Error bars depict standard error of the mean.
Figure 6.
Figure 6.
Reliance per day in Experiment 1. The black line denotes actual automation performance (75% correct). Error bars depict standard error of the mean.
Figure 7.
Figure 7.
Baseline expectations and perceptions per day in Experiment 1. The symbols on the y-axis denote the manipulation given to each group through an explicit statement of 60% or 90% reliability. The black line denotes actual automation reliability (75% correct). Error bars depict standard error of the mean.
Figure 8.
Figure 8.
Performance in total task points per day in each experiment. Error bars depict standard error of the mean.
Figure 9.
Figure 9.
Compliance per day in each experiment. The view of the lower-than group’s data is obstructed by the higher-than group’s data for Experiment 1 only. The black line denotes actual automation performance (75% correct). Error bars depict standard error of the mean.
Figure 10.
Figure 10.
Reliance per day in each experiment. The black line denotes actual automation performance (75% correct). Error bars depict standard error of the mean.
Figure 11.
Figure 11.
Baseline expectations and perceptions per day in each experiment. The symbols on the y-axis denote the manipulation given to each group through an explicit statement of 60% or 90% reliability. The black line denotes actual automation performance (75% correct). Error bars depict standard error of the mean.
See this image and copyright information in PMC

References

    1. Barg-Walkow LH (2013). Understanding the role of expectations on human responses to an automated system (Unpublished master’s thesis). Georgia Institute of Technology, Atlanta, Georgia.
    1. Barg-Walkow LH, & Rogers WA (2014). Introducing an automated system: Effects on perceptions and use of the system. Proceedings of the Human Factors and Ergonomics Society 2014 Annual Meeting (pp. 1501–1505). Santa Monica, CA: Human Factors and Ergonomics Society.
    1. Bliss JP (1997). Alarm reaction patterns by pilots as a function of reaction modality. The International Journal of Aviation Psychology, 7(1), 1–14.
    1. Bliss JP (2003). An investigation of extreme alarm response patterns in laboratory experiments. Proceedings of the Human Factors and Ergonomics Society 47th Annual Meeting (pp. 1683–1687). Santa Monica, CA: Human Factors and Ergonomics Society.
    1. Bliss JP, & Dunn MC (2000). Behavioural implications of alarm mistrust as a function of task workload. Ergonomics, 43(9), 1283–1300. - PubMed

Publication types

LinkOut - more resources