Joint modelling of brain and behaviour dynamics with artificial intelligence

Abstract

Artificial intelligence has created tremendous advances for many scientific and engineering applications. In this Review, we synthesize recent advances in joint brain-behaviour modelling of neural and behavioural data, with a focus on methodological innovations, scientific and technical motivations, and key areas for future innovation. We discuss how these tools reveal the shared structure between the brain and behaviour and how they can be used for both science and engineering aims. We highlight how three broad classes with differing aims - discriminative, generative and contrastive - are shaping joint modelling approaches. We also discuss recent advances in behavioural analysis approaches, including pose estimation, hierarchical behaviour analysis and multimodal-language models, which could influence the next generation of joint models. Finally, we argue that considering not only the performance of models but also their trustworthiness and interpretability metrics can help to advance the development of joint modelling approaches.