Compensating for Sensing Failures via Delegation in Human-AI Hybrid Systems

Abstract

Given the increasing prevalence of intelligent systems capable of autonomous actions or augmenting human activities, it is important to consider scenarios in which the human, autonomous system, or both can exhibit failures as a result of one of several contributing factors (e.g., perception). Failures for either humans or autonomous agents can lead to simply a reduced performance level, or a failure can lead to something as severe as injury or death. For our topic, we consider the hybrid human-AI teaming case where a managing agent is tasked with identifying when to perform a delegated assignment and whether the human or autonomous system should gain control. In this context, the manager will estimate its best action based on the likelihood of either (human, autonomous) agent's failure as a result of their sensing capabilities and possible deficiencies. We model how the environmental context can contribute to, or exacerbate, these sensing deficiencies. These contexts provide cases where the manager must learn to identify agents with capabilities that are suitable for decision-making. As such, we demonstrate how a reinforcement learning manager can correct the context-delegation association and assist the hybrid team of agents in outperforming the behavior of any agent working in isolation.

Keywords: delegation; entity detection; reinforcement learning; simulated sensing.

Figure 1

Diagram of manager and agent observation, selection, action, and environment update.

Figure 2

Manager deep reinforcement learning architecture diagram.

Figure 3

Adverse driving condition contexts (red and blue rectangles representing cars). (a) Base case, (b) Foggy weather, (c) Night/low light, and (d) Night with fogginess.

Figure 4

Sensing with masks for adverse driving condition contexts (blue rectangle representing a car and white rectangles representing masks). (a) Base sensing case, (b) Sensing in foggy weather, (c) Sensing night/low light, and (d) Sensing night with fogginess.

Figure 5

Mean observed color for color-based error condition contexts. (a) Default, (b) Color with night/low light, (c) Color with fogginess, and (d) Color with night and fogginess.

Figure 6

Sample driving environment including rectangular buildings, sidewalks, and cars (blue, cyan, red, and green rectangles) with shortest path trajectory. (a) Main environment and (b) Sample path (black and green lines representing road segments and shortest path in red).

Figure 7

Key interaction zones for driving scenario(s), with cars represented by blue, red, and green rectangles. (a) Four-way intersection and (b) T-intersection.