Towards Agency in Human-AI Collaboration

Abstract

Artificial Intelligence (AI) is increasingly evolving from a tool for automating repetitive tasks to an intelligent agent actively engaging in dynamic interactions with humans. As AI becomes more integrated into collaborative contexts, it is essential to examine the factors that shape human-AI interaction. Central to this collaboration is AI agency-the capacity for action and effect-a concept that has remained largely peripheral in existing research. This paper addresses this gap by proposing a comprehensive design space for reasoning about agency in human-AI collaboration. We introduce the high-level perspectives of distribution, modeling, and attribution to outline key dimensions that inform the design of agency in such systems. Our methodology combines a literature review with expert interviews to consolidate existing concepts and surface new insights. To exemplify the capacity of our framework, we reason about three mixed-initiative systems through the lens of our conceptual model. Finally, we identify future directions and critical research gaps in this emerging area.