How transparency modulates trust in artificial intelligence

Abstract

The study of human-machine systems is central to a variety of behavioral and engineering disciplines, including management science, human factors, robotics, and human-computer interaction. Recent advances in artificial intelligence (AI) and machine learning have brought the study of human-AI teams into sharper focus. An important set of questions for those designing human-AI interfaces concerns trust, transparency, and error tolerance. Here, we review the emerging literature on this important topic, identify open questions, and discuss some of the pitfalls of human-AI team research. We present opposition (extreme algorithm aversion or distrust) and loafing (extreme automation complacency or bias) as lying at opposite ends of a spectrum, with algorithmic vigilance representing an ideal mid-point. We suggest that, while transparency may be crucial for facilitating appropriate levels of trust in AI and thus for counteracting aversive behaviors and promoting vigilance, transparency should not be conceived solely in terms of the explainability of an algorithm. Dynamic task allocation, as well as the communication of confidence and performance metrics-among other strategies-may ultimately prove more useful to users than explanations from algorithms and significantly more effective in promoting vigilance. We further suggest that, while both aversive and appreciative attitudes are detrimental to optimal human-AI team performance, strategies to curb aversion are likely to be more important in the longer term than those attempting to mitigate appreciation. Our wider aim is to channel disparate efforts in human-AI team research into a common framework and to draw attention to the ecological validity of results in this field.

Keywords: artificial intelligence; explainable AI; human factors; human-AI teams; human-computer interaction; machine learning; transparency; trust.

Figure 1

Scale of user attitudes toward AI in human-AI teams

Figure 2

User trust in automation after witnessing system failures (A) Five possible trust trajectories over time. Notice that the default attitude toward automation is generally one of high trust that falls by some measure in response to seeing a system err. The vigilant user of AI recalibrates their initially unrealistic estimate of a system’s capabilities gradually, but not to the point where their attitude becomes aversive. (B) The hypothesized role of transparency in trust calibration

Figure 3

Trust versus reliability