Inferring an Observer's Prediction Strategy in Sequence Learning Experiments

Abstract

Cognitive systems exhibit astounding prediction capabilities that allow them to reap rewards from regularities in their environment. How do organisms predict environmental input and how well do they do it? As a prerequisite to answering that question, we first address the limits on prediction strategy inference, given a series of inputs and predictions from an observer. We study the special case of Bayesian observers, allowing for a probability that the observer randomly ignores data when building her model. We demonstrate that an observer's prediction model can be correctly inferred for binary stimuli generated from a finite-order Markov model. However, we can not necessarily infer the model's parameter values unless we have access to several "clones" of the observer. As stimuli become increasingly complicated, correct inference requires exponentially more data points, computational power, and computational time. These factors place a practical limit on how well we are able to infer an observer's prediction strategy in an experimental or observational setting.

Keywords: Bayesian models; prediction; sequence learning; stochastic processes.

Figure 1

The quintessential experiment to reveal an observer’s prediction strategy: observations are shown to the observer, who then tries to predict the next observation. This happens repeatedly for as many trials as the observer can stand.

Figure 2

Two example order-R Markov models. Order 0 (top) and 1 (bottom) with the finite alphabet $\mathcal{A}=\{0,1\}$.

Figure 3

The average error in ${\varphi }_{ngram-argmax}$ parameter inference for n-gram argmax prediction strategy and ${\varphi }_{ngram-average}$ for the n-gram average prediction strategy shows that the inference seems to perform perfectly. For n-gram average, we see that every combination of parameters was able to produce perfect estimates for ${\varphi }_{ngram-average}$. For n-gram argmax, the sample sizes were 7 for each pair corresponding to $R=2,6$ and 10 for $R=4$. For n-gram average, the sample sizes were 9 for each $(R,N)$ pair.

Figure 3

The average error in ${\varphi }_{ngram-argmax}$ parameter inference for n-gram argmax prediction strategy and ${\varphi }_{ngram-average}$ for the n-gram average prediction strategy shows that the inference seems to perform perfectly. For n-gram average, we see that every combination of parameters was able to produce perfect estimates for ${\varphi }_{ngram-average}$. For n-gram argmax, the sample sizes were 7 for each pair corresponding to $R=2,6$ and 10 for $R=4$. For n-gram average, the sample sizes were 9 for each $(R,N)$ pair.

Figure 4

Surface and contour plot of log likelihood of the n-gram argmax prediction strategy show a peak at some combination of parameters. On the x and y axes are parameters ${\varphi }_{ngram-argmax}$ (the ratio of $\beta$, the probability of dropping an observation, to $\alpha -1$, the concentration parameter subtracted by 1) and $\gamma$ (the regularization term in the prior over models), and on the z-axis is the log likelihood of the observer model for a string of inputs, averaged over infinite identical observers. It is difficult to see in the pictures, but there are ridges in the average log likelihood as a function of ${\varphi }_{ngram-argmax}$, which we still cannot explain.

Figure 5

Surface and contour plot of log likelihood of the n-gram average prediction strategy show a much smoother surface than n-gram argmax. On the x and y axes are parameters ${\varphi }_{ngram-average}$ (the ratio of $\beta$, the probability of dropping an observation, to $\alpha$, the concentration parameter subtracted by 1) and $\gamma$ (the regularization term in the prior over models), and on the z-axis is the log likelihood of the observer model for a string of inputs, averaged over infinite identical observers. This appears to be a much nicer surface to optimize over, though it is not without its ridges.