Behavioural signatures of backward planning in animals

Abstract

Goal-directed planning in behavioural and neural sciences is theorized to involve a prospective mental simulation that, starting from the animal's current state in the environment, expands a decision tree in a forward fashion. Backward planning in the artificial intelligence literature, however, suggests that agents expand a mental tree in a backward fashion starting from a certain goal state they have in mind. Here, we show that several behavioural patterns observed in animals and humans, namely outcome-specific Pavlovian-to-instrumental transfer and differential outcome effect, can be parsimoniously explained by backward planning. Our basic assumption is that the presentation of a cue that has been associated with a certain outcome triggers backward planning from that outcome state. On the basis of evidence pointing to forward and backward planning models, we discuss the possibility of brain using a bidirectional planning mechanism where forward and backward trees are expanded in parallel to achieve higher efficiency.

Keywords: Pavlovian-to-instrumental transfer; computational modeling; differential-outcome effect; planning; reinforcement learning.